Tag Archives: Palringo

Mistakes

Posted on by under Business, Tech

We all try, in our own ways, to be perfect. But everyone makes mistakes.

The past couple of months have been an interesting time at Palringo. A few times a year the offices converge on Newcastle for a “reset meeting” where we discuss our plans for the next 3-6 months, re-evaluate our current commitments, and scrap anything we no longer believe in.

The “Invite Centre” was originally conceived by Martin Rosinski approx. 6 months ago and presented at one of these meetings. I was tasked with making the largest user-facing project of 2013 (so far) a reality. The “Invite Centre” is how we plan to grow the Palringo community – and is heavily related to the Reputation and Achievements system I have detailed before. The “Invite Centre” would be the “big ticket item” for Palringo iOS v6.4 and Palringo Android v5.10.

Palringo iOS v6.4 was released on Monday 2nd Sept, 2013, our previous major release, Palringo iOS v6.2, was released on Monday 22nd July, 2013 – with a follow up bug-fix release, v6.2.1, on Monday 29th July.

Some background; all major releases will have up to 2 big ticket items – these are tasks that require a lot of development and testing time, and may not be apparent to the user. For v6.2 this was a transition to CoreData. A change of this scale will always result in bugs; the simple mantra of “the bigger the change, the more bugs there will be” always holds true. You can never test every combination, or find every bug – this is part and parcel of software development, and shouldn’t be a surprise to anyone reading this – our game is to minimise issues, and begrudgingly accept that you’ll never eradicate every idiosyncrasy, and celebrate when you do finally manage to terminate that one little bastard that has been haunting you for months.

The need for v6.2.1 was apparent early on, both from the reports we were getting back from TestFlight – an iOS crash reporting system – and our in-app Support team. The scope for the bug-fix was defined, fixes found and made, and the update released 7 days later – an impressive feat, especially when taking in to account the Apple Review Process, and the time it takes to get an app reviewed.

For all parties, v6.2.1 was an improvement – but still had more issues than pre-6.2 releases. This is another issue with updating any core functionality for a piece of software. Once you get the big bugs out of the way, you spend years continually polishing the code, ensuring it is running as best as it can. This level of polish is not possible to replicate in a month, or two, when you have to change something significant.

The “Invite Centre” is getting closer, so we barrel on through, deciding not to make another bug-fix release.

Feedback from our Support Dept is that the users are living with the current issues that still arise in v6.2.1, which I incorrectly assume means they are happy. They are not.

This was my first mistake.

We continue, and as the week becomes two, and then three, the complaints from our Support Dept increase. It is already the 20th August, with v6.4 meant to be our “August Release”, there are minimal signs of submission soon.

A meeting is called and we – myself, the devs, and QA – detail the situation, explaining that we aren’t happy with the current state of v6.4 and the fixes we have included. We don’t believe the experience will be any better (“We’ve not so much fixed the bugs, as just moved them to other places.”) and that there are a number of known bugs. One of our external testers continually experiences this bug, but cannot find a way to reproduce, and neither can we when plugged in to the debugger. We diagnose this bug as being “severe but infrequent”, and decide that we can live with it. We agree, in part due to the ever increasingly vocal complaints from our users, that we would fix the high priority bugs, and submit the app in close to its current state, acknowledging it isn’t perfect, but will be better when make these fixes.

This was my second mistake.

Once we have made all the agreed fixes, we submit to Apple.

We spend the week between submission and release attempting to reproduce the “blank messages” bug (as it is now known), to no avail. The App is approved over the weekend – Apple didn’t find it either – and I wait for Support to confirm that nothing has happened over the weekend for them to insist on a delay.

While I’m waiting, we finally reproduce the bug. It only occurs under very specific conditions. Firstly, you had to have minimal memory available – which in Palringo happens if you are in many, many groups, receiving hundreds of messages a minute – and secondly you receive a message (group or PM) when not viewing the messages tab.

My choice is simple;

  • Pull the approved app, which we know is an improvement for most, and hope we can find a fix in a short enough window of time to please the community that are increasingly vocal in their complaints about v6.2.1, or,
  • We release, and bug fix if required.

It is my decision – and the fact we can now reproduce the bug does not matter. The submission passed our locks with us knowing the bug existed, without a caveat of “if we can reproduce, we’ll pull”. This is different to a bug you suddenly find before a release – that would cause a delay for at least as long as it took to triage. Fixing bugs is an on going aspect of software development, when you find out how to fix them is irrelevant, if you’ve decided that you are happy with said bug being present.

I release the app.

This was my third mistake.

It would ultimately transpire that 99% of our user base would not be affected by this bug, and that v6.4 is a noticeable improvement – but the 1% it does, it is a continually reoccurring nightmare.

We get reports of users experiencing the bug every 4 minutes, that the app is unusable, that they need to downgrade. We cannot roll back the app – Apple don’t let you – and we can’t re-submit v6.2.1 due to the approval delay. We add text to the “What’s New” section of the App Store Description, telling high-usage users to not upgrade to this version of the app, as well as passing out the message in-app.

We start frantically fixing the bug. Either there is a problem in the locks, or, we made a mistake.

“Release early, release often” is how we manage QA, nightly builds are increasingly common as we progress through the release cycle. Our bug-fix process barely differs from a standard cycle, and within a few hours we have a proposed fix and have pushed a beta release to our testers, with steps to re-create the situation. “Negative Testing” is some of the most time consuming software testing you can do – and is strongly related to the concept of proving a negative – and so all you can do is state that “these steps no longer seem to cause the problem(s)”. You can never really explicitly state the bug has be fixed.

I call a meeting Tuesday morning for Technical and QA, for 30 minutes we discuss just how we got here, why we are in the situation, have a full debrief, and discuss what we need to change. We have faith in our process, we admit, understand, and accept that the bug was misdiagnosed – we agree that, if we had to do it again, we would. We get back to work.

Our testers report back, things seem better…but different. We have new bugs. We apply yet more plasters, but it becomes clear that this is a fundamental issue with the framework.

The Technical Lead consults with me, and expresses his desire to rip out the back end, and completely re-implement the view. We decide that this is the best way forward.

A “functional but not pretty” beta is released with the old framework ripped out, and the stability improvements are apparent from the get go. A by product is that, because we removed the view completely, that specific bug can no longer exist.

Many more interim releases are made, and late Friday we make a final RC release to our beta team, and agree to submit to Apple in parallel. At the time of writing, we are awaiting approval.

—

So, what have I learnt?

  1. The balance between fixing bugs vs. new features is even more delicate that previously imagined; and it must be a balance, you cannot sacrifice one for the other.
  2. It is always better to delay something slightly, than make an action you cannot undo.
  3. Confirm your diagnosis of bugs is still true if presented with new evidence or data.
  4. If a bug is frequently encountered by one person, it will definitely be encountered by the many.
  5.  Everyone makes mistakes – but it is unforgivable to not learn from them.

Gamification: Game Playing and Fiero

Posted on by under Gamification

This is part two of the series, to start from the beginning, click here.

Humans have enjoyed playing games for thousands of years; archaeologists have found extensive evidence of dice-like games being played in region of South Eastern Iran approximately 3,000 years ago, and board games – like senet – being enjoyed by the Ancient Egyptians some 5,000 years ago.

Considered by some to be an early form of escapism, and a simple way of coping with the harsh realities of life, games such as Jacks (or Knucklebones) were invented by the Lydians and theorised to have been utilised to cope during times of famine.

But games are not limited to this simplistic, traditional view. Most human interaction within any society involves many aspects of game playing. Eric Berne‘s 1964 book “Games People Play: The Psychology of Human Relationships” extensively details many different types of human interaction from a variety of perspectives, such as how differing power dynamics, or roles such as adult to adult, or adult to child, affect each players strategy in the game. Gamification seems to stand to confirm Kurt Vonnegut’s review of Games People Play, where he stated that; “…the good Doctor has provided story lines that hacks will not exhaust in the next 10,000 years”. The identification of these games provided an early insight in to what would eventually become the basis for Palringo’s Reputation and Achievements System, and how a competitive system based on these principles can be a very powerful engine for growth.

In the introduction I briefly mention the concept of fiero. Fiero is a defined by Jane McGonigal in her book “Reality is Broken: Why Games Make Us Better and How They Can Change the World” as a word to:

“…describe an emotional high we don’t have a good word for in English [Fiero being Italian for ‘Pride’]. Fiero is what we feel when we triumph over adversity. You know when you feel it – and when you feel it […] we all express fiero in exactly the same way: we throw our arms over our head and yell.”

Fiero is the basis for all game systems. You win, you feel good, you lose, you feel bad…and then you’re annoyed by the obvious display of fiero by the victor, and you try even harder to win – after all, everybody loves to win.

Simply put, fiero breads competition and powers our innate urge to win.

A key consideration that must be made when designing any game system is the difficulty to achieve fiero. While no-one actually likes losing, always winning also quickly becomes bland – after the initial hits of fiero dissipate.

Casino Poker has a number of variables, but to simplify this example we will focus on two factors;

  1. Skill of the Player
  2. Disposable Cash

Casinos spend significant time and resources to ensure their players are sitting at tables where they all have a similar level of skill and cash to ensure the best game experience for everyone. There is little fun when there is no challenge, and easily winning (or always losing) causes the same level of disenfranchisement as never having any risk. A multi-millionaire winning a $100 bet feels much less fiero than a student on minimum wage winning the same bet. This is also true for the other players, if you are at a table with a person that simply doesn’t care about the wagers that are being made, then the fiero you experience when winning is massively reduced.

These systems require the players to “buy-in”, much like the social contract that exists stating that participants should not lie when playing ‘Truth or Dare”. Managing this belief in the system – in the poker example, promoting good player to higher tables – is important to ensure continued competition.

Foursquare operates a system that distributes points to users when they check-in to bars, clubs, restaurants, and so on. These check-ins are also supported by a series of achievements: checking in to a bar on a week day evening, or going to the gym 10 times in a month. Xbox LIVE (XBL) Gamerscore – often considered to be the first and best video game meta-scoring system – has a similar system of points and achievements.

The big difference between the two however is that a player’s XBL Gamerscore is counted forever, where Achievements you unlocked in 2007 are still counted towards your score in 2013, and can never reduce. In contrast, Foursquare only counts points for a rolling 7 days, where points I earn today will be counted towards my score until next Monday. In this regard Palringo Reputation is closer to Gamerscore, where points are counted forever, but with the added complexity of being able to lose reputation by losing achievements.

Foursquare is tailored to be much more competitive, constantly informing you of your friends activity – with notifications when they check in – as well as any new achievements they unlock. You can always see how you rate against your friends, in a simple leaderboard system where having the most points means you’re #1. Competing on a 7 days basis, where simply being inactive takes you out of the running, means that the challenge is always fresh, in an ever changing landscape, where victory is only temporary, and must be continually fought for.

The Palringo Reputation and Achievements system primarily serves two types of fiero. The first (bigger, and rarer) via Reputation, and the second (smaller and more frequent) via Achievements. To build on the previous two examples, Reputation is a meta-score to provide a quick, clean, and cross-culture mechanic to compare two or more users. Our users use their Reputation as their primary means to compare, compete and brag about their standing within the Palringo Community. Achievements are used as guides to inform users on how to increase their Reputation, as well as encourage behaviour that we deem is suitable for Palringo – for example, running multiple groups, with a high number of users, as detailed by our “Crowd Sorcerer” achievement.

In the next part of this series, I will expand on just how Palringo uses gamification, and fiero, to guide user behaviour and cement our place as the de-facto massive Group Messaging Application in the world.

Gamification: An Introduction

Posted on by under Gamification

Gamification is a buzzword for the application of various aspects of positive psychology in conjunction with game-like elements used to guide people’s behaviours when performing tasks. While this is by no means solely utilised by software, many modern software applications have utilised its theories extensively, especially video games such as World of Warcraft.

We have an entire generation that grew up playing video games, being born in 1989 I have played games my entire life, and have seen first hand their progression from side-scrollers like Super Mario to the insanely complex high-end graphics of series such as Half Life. Even with the massive increases in processing power, memory and storage, gameplay has same stayed roughly the same. Most games follow the same pattern, you start a level, you play, you fail, you try again, you fail, you try again, you fail, you try again, and again, and again, and then you succeed. Then you start the next level.

This wash-rinse-repeat style of gameplay coupled with regular praise creates a very tight feedback loop and is even in some of the oldest video games.

Jane McGonigal in her book “Reality is Broken” frequently states that praise (or fiero), coupled with an increasing difficulty and frequent testing, creates a quickly addictive environment. It is the sense of accomplishment, traceable progression, and an attainable next step that keeps people coming back for more.

Tetris has a very simple premise. You score points by completing a row of blocks using various shapes that are generated (fall) in a pseudo-random order to progress. Every time you complete a row you are awarded points, the row flashes and then disappears (your small-reward), allowing you to use the re-claimed space to complete more rows. When you have scored enough points by removing enough rows you advance to the next level (traceable progression), which clears the whole screen (your big reward), and increases the difficulty (genuine testing). Tetris is perhaps one of the most obviously repetitive games in existence but Tetris combats this with two further sub-systems. A high-score leaderboard displaying the number of points you have earnt, allowing you to track your progress between sessions, and increasing the number of points you earn for each successful clearance of a line on the harder levels (genuine accomplishment), making a clearance much more satisfying on Level 9, than on Level 1.

It is the supply of praise – specifically the management of craving vs. frequency of genuine accomplishments – that must be utilised if you wish to create a system that truly works.

B. F. Skinner, an Edgar Pierce Professor of Psychology at Harvard University devised the “Operant Conditioning Chamber” – commonly, and for the rest of this series, referred to as the “Skinner Box” – whilst a Masters student at Harvard in 1930. An extremely simplified example of this could be a rat in a box, with only a red light and a button. When the red light flashes, the rat has a few seconds to press the button and be dispensed food. If the light has not flashed, then pressing the button does nothing. Over time, the rat learns to associate the feedback loop of Light => Press Button => Food and will act accordingly, and it is this basic premise that fuels most gamified systems.

User performs desired action -> User gains reward
User performs undesired action -> User receives nothing, or is punished.

Using this premise within a video game environment is simple – games as a whole are by their very nature built around similar constructs which most are familiar with, be it from playing Tag in the playground at school as a child, to card games like Poker – and so polishing these functions requires less relative effort.

Applying these theories to a software application is considerably harder and requires the application to be suitable. A word processor, for example, is unlikely to benefit – although one could argue that even Word has many of these elements. It is my belief that the more social an application is, the more suitable and powerful gamification mechanisms are.

Palringo utilises two complementary systems. A points-based scoring system referred to as “Reputation” or “Rep” which is displayed to all as a Level (for example, “Level 1”) and a goal-based system called “Achievements“; these are badges that you earn, and again are displayed to everyone. At a conceptual level, users receive points based on behaviour that we deem appropriate, and are deducted points for behaviour we do not. Various achievements are used as both long and short term goals to give users a guide as to what to aim for next, thus further helping shape their behaviour.

I will go in to more detail about how Palringo has continues to successfully operate and expand this system as this series progresses but before I continue I feel I must explicitly state what this series is not, and will not cover. It will not cover any detail of the Palringo system that is contained within a black-box – i.e. it does not describe all factors of our Reputation system. Some factors however will be mentioned explicitly, many may be alluded to, lots will be omitted in their entirety, and there may be the occasional false flag. This is to protect the integrity of our system, and reduce the possibilities for exploitation. This series will also not contain any detailed technical descriptions, or examples of code.

What this series will cover are the themes, behaviours, and insights that are required to create a successfully gamified system – one that can, if required, save a company.

—

Part Two of this series: Game Playing and Fiero.

Bookshelf I – Abundance: The Future Is Better Than You Think

Posted on by under Bookshelf, Business, Future, Tech

After months of trying to finish off my Christmas Reading List – success eventually arriving in late February – I finally read “Abundance: The Future Is Better Than You Think” by Steven Kotler and Peter Diamandis. For those that haven’t yet read it, ‘Abundance’ extols – to a manifestoesque degree – a new hybrid of ‘Philanthrocapitalism’ and ‘Technophilantropy’, promoting – and in many cases, providing convincing proof – that using technology a rising tide really can lift all boats.

Health Care (23andMe), Energy (Elon Musk’s SolarCity, et al), Food (Vertical Farms and hydroponics), Freedom of Speech and Association (Twitter, Facebook, etc – Palringo being a noted absence), and Micro-finance (Kiva.org) being just a few of the wide range of topics covered; ‘Abundance’ makes the claim that with a new way of thinking, with a focus on global rather than local solutions, and providing technology that stands to save us all, we may have the ability to not just provide food, water, and shelter to 9 billion people – but to provide these at a standard that would befit the average European today. This can be accomplished, ‘Abundance’ claims, by utilising the rate of growth associated with exponential technologies.

‘Abundance’ proposes a new model, based heavily on Maslow’s ‘Hierarchy of Needs‘ named – cunningly – the ‘Abundance Pyramid’. Each part of the book relates to each of the levels in the Pyramid and how existing companies are working towards providing each of the requirements. I won’t replicate the Pyramid in full here, however ‘Part 3: Building the Base of the Pyramid’ is focused on how to provide: Communication (via smartphones, and unfiltered universally accessible internet access); clean, reliable, and accessible water; and a sustainable food supply, with a heavy focus on GMOs.

While ‘Abundance’ does pay lip service to the potential problems that may arise with smarter technology, it is written – like any manifesto – from an optimistic view point, and largely downplays any of the potential problems. This is in stark contrast to ‘The Net Delusion: How Not to Liberate the World‘, by Evgeny Morozov that extols the view that smarter technology, whilst providing many positives, also gives corrupt regimes access to very highly targeted data and that it is only a matter of time before these regimes utilise technology similar to that which runs Google’s AdSense platform for less-than-good purposes.

As the book progresses, Peter Diamandis’ influence becomes more prevalent, with many mentions of both his ‘Singularity University‘ and the ‘X-Prize‘ culminating in his belief that incentive prizes, like the X-Prize, are the only way to accelerate the progress that we are currently making. He very may well be correct, however the proof provided – there have, so far, only been 3 completed X-Prizes – seemed to be lacking for a statement of that calibre. However, given Dr Diamandis’ track record, I don’t expect him to be far off the mark.

Overall, the book is a very interesting read, highlighting some key areas, and has definitely challenged – and indeed changed – some of my previously held perceptions, and for that reason alone, I’d highly recommend it.