When Your Awesome Feature Turns Out to Be a By-Design Bug

I want to share a lightbulb moment I had recently. I love my work as a product manager in SysAid, being empowered by data from one end, and working closely with SysAid customers from the other end.

Each interaction with customers is an opportunity to listen to the customers’ needs as well as share some fresh ideas, and as a result – create meaningful features.

Not long ago, we had a request from a customer to add specific functionality to our notifications feature.

Here’s how the story began…

Our new awesome feature

Based on the request of one of our customers, new functionality was inserted to enhance an existing feature regarding notifications.

The feature was very specific: it allows agents to get notified about specific actions when their users modify attributes in SysAid.

The whole Product department was so excited, that we even added a special shout-out to the customer inside the product, to thank them for encouraging us to add this functionality. And the customer truly appreciated this gesture.

Going back to the Beta stage

This feature request was initiated from feedback that we received from a customer via the support department. We inserted it into our product backlog and after analyzing it, we interviewed the customer.

At this point we started to validate this requirement with other customers as well, trying to understand the impact of this new feature on our customers’ ecosystems. We realized that this functionality has a positive ROI, especially when we compared the development efforts against the impact of existing and potential customers.

For us, it was a simple and straightforward formula: happy customers + positive ROI = everybody is happy. A classic win-win situation.

However, as the Beta progressed, we understood that we might be missing something…

The support department reported that a certain number of customers included in the Beta complained about getting double notifications, which they did not get previously. My team and I then reached out to all the customers who reported the issue. Some customers told me that they’re confused, while others reported that it was annoying. But one customer actually said that it was highly useful!

The range of feedback received matched the user adoption statistical results, where some customer segments significantly increased adoption, while others showed a significant decrease. We were surprised about this inconclusive result. But it was conclusive for us, some customers found our feature useful and others considered it as a bug.

Is it possible that we had created a by-design bug?

Considering that:

• We’re a top leading company in our domain, we’re customer centric, always thinking of our customers from all angles across all business units.
• Our scale is high, consisting of thousands of customers and hundreds of thousands of users, we are used to the high impact of even the smallest change.
• Our product department is well organized, following the most updated best practices as well as using state of the art frameworks that help us to orchestrate the entire product ecosystem.
• Our technologies are cutting edge and know how to fulfill (almost) all product requirements.

And yet, all the facts above are irrelevant when answering our question.  The answer will always be “Absolutely yes!” We indeed created a by-design bug.

Is it possible to avoid creating a by-design bug?

When it comes to by-design bugs, it cannot always be avoided. But it’s certainly possible to reduce the chances by asking yourself the following questions:

• Is this a new feature? Perhaps it is just new functionality to an exciting feature, and you need to then consider the impact to your customers already using it.
• Dive (again) into the stats. What is the current adoption rate that you already have in this feature? What is the expected adoption?
• Are all scenarios covered in your product requirements document (PRD)? Have your well- written scenarios been applied to all users? All segments? What about existing vs. new? How are these users going to be aware of functionality when it comes to an existing feature? For example, a new checkbox. And if so, will this checkbox be selected by default and automatically introduce this functionality to all users? What will be the impact of their workflow in each case, and how are you going to introduce it from a UX perspective?
• In which stage are you going to introduce this feature? Alpha? Beta?
• Have you considered protecting this feature with a flag? This will allow you to introduce the feature only to a small group that you  choose, giving you time to gather their  feedback before deciding  to launch it to all customers.
• And the list goes on…

Turning lemons into lemonade

Once we understood the case, the solution was simple. The ‘fix’ implemented was the adding of a checkbox that will allow our customers to decide whether to use it or not. The default was unchecked, so that this new functionality was not introduced by default to existing customers. However, to the segment that found it useful based on our research and their feedback, we left it ‘checked.’

One month after the General Availability, our stats showed high adoption rates and happy customer feedback! We realized we had fixed the problem, we were glad we had learned a thing or two, and especially how to turn this lemon into lemonade.