I'm part of the team that has recently launched a web app called Dunked. Dunked is a free simple-to-use online portfolio for creative types. As of May 2013, we are in public beta; continuing to build out features as well as making incremental improvements based on user feedback.
In this article, I am going to discuss some of the reasons I believe that a beta launch is important to the development of your product. I will also discuss how we, at Dunked, are going about the process of monitoring and gathering user feedback. Finally, I will discuss how were are going about implementing changes based on user feedback as we continue to build the product out.
Why I Like Beta Launch
I personally believe that having a beta launch is a really great idea. Of course, it’s inadvisable to release something which is riddled with bugs. It’s also a bad idea to release a product that isn’t ready for public consumption. It’s far better to release the product once it has the necessary core features and is free of bugs.
Google is probably one of the more well known companies that initially offers products as a beta release. Gmail is a great example. Gmail was in private beta from April 2004 to February 2007, when it was made available to the general public. Even then, it still had the "beta" label. Gmail wasn't officially taken out of beta for another two years. Having it as a beta product enabled Google engineers to continue enhancing Gmail based on user data as well as to test various additions.
Perfect just doesn't exist within software development
I recognize that it can be tough letting users see a product before you believe it to be perfect. But perfect just doesn't exist within software development. You will always need to iterate over a product in a constant effort to make improvements. I believe that by releasing a beta version of a product, you put yourself in a position to best offer users what they want. You may have assumptions that are just flat out wrong. By operating in beta, you can quickly pivot to improve your product to your users' needs.
In taking Dunked to launch, we felt that it was crucially important to identify and establish the differences between must-have features and nice-to-have features. Must-have features are the features which shape, and essentially, make your product. Must-haves are the features on which you should focus your time pre-launch. Once you've completed your list of must-haves, launch. After launch, you can tackle the nice-to-haves, fix any bugs that arise, and make adjustments based upon user feedback.
Monitoring and Gathering User Feedback
As we continue to build, user feedback is like gold. Users tell us when we're doing things great; when we're doing things wrong; and when we're being just plain stupid. Because user feedback is so valuable, it is important that you have a system in place prior to launch so that you can monitor and gather feedback. You must also establish a system to act upon the feedback that you receive.
Monitoring and gathering feedback is no easy task. Users are all different, with varying preferences in how they go about expressing their opinions. Therefore, you need to have a system in place to actively monitor a variety of channels of feedback. We've opted to use Desk. It serves as our central hub collecting all tweets directed at our @DunkedHQ Twitter handle, comments on our Facebook page and emails sent via our support contact form (this form is linked to from within the app itself). It enables me to log into a single location, where I can review and respond to feedback in the order it was submitted. This is a win-win for us and for users. Users can express their opinions however they feel most comfortable, and we can very easily access all feedback in a single location.
That handles current users, but what about the people that just delete their accounts without offering any feedback. Know that accounts will be closed, and do your best to collect information from users closing their accounts. At Dunked, we direct the user to a quick "exit" survey once they have completed the account deletion process. The survey is completely optional and can be completed in under five minutes. Most folks have been willing to fill it out, providing us with additional feedback on why they deleted their account. We've used Wufoo to handle our survey needs because they make it really dang easy.
Using User Feedback
Now that we have a method of monitoring and gathering feedback, what should we do with it? For Dunked, we have a pretty simple approach. We use a to-do list within Basecamp to house the feedback we collect. Whenever a user writes in with a suggested improvement or requests a missing feature, we add it to our Wishlist/Suggestions to-do list. This list also contains our previously mentioned nice-to-have features. Not all the items that get added to the list will be added to Dunked, but it helps us to monitor the suggestions. Common feature requests are noted, moved to the top of the list, and are often included in our code sprints, which I will discuss later.
Unfortunately, the Dunked platform hasn't been perfect, so we do occasionally get bug reports. We maintain a separate bug list also within Basecamp. Any item added to the bug list is considered a top priority and is fixed as quickly as possible.
As I previously mentioned, we are continuing to build out core features for Dunked, but we feel it is important to act upon the feedback provided by early users. Because users have been so great in providing us with feedback, we want to implement their suggestions as time permits. We do this through daily code sprints. Once or twice a week, we will go through our Suggestions/Feedback list, and pick a few items that can be completed in a single day of coding. We complete those items, test on our development server, and then push to the production server once we are happy with the code.
To illustrate how user feedback has improved Dunked, consider the following example. For image uploads, we created a simple, click-to-upload button. Clicking the button launched the file browser on the user's computer enabling them to browse to and upload images for their projects. It worked perfectly for us. Users, however, had some issues.
There are two basic options on how to handle image uploads: click-to-upload or drag-and-drop to upload. As it turns out, we (Team Dunked) each prefer click-to-upload. We didn't envision that users would want drag-and-drop uploads. In gathering feedback, it became evident that we had room for improvement in image uploads. We added "Drag-and-drop uploads" to our Wishlist/Suggestions list. After multiple requests, drag-and-drop uploads was added to one of our early code sprints. Now when you want to upload images to a project, you can click-to-upload or drag-and-drop.
This is a rather simple example of how users used Dunked differently than we envisioned. We never considered that users would be put off by our traditional method of image uploads. In monitoring, gathering, and acting upon user feedback, however, we were able to improve the Dunked for our users.
The Limitations of a Beta Release
Now that I've explained a bit about our approach to a beta release and why I think beta releases are great, I must discuss some of the mistakes and limitations that could plague a beta release. The biggest mistake that one could make in a beta release is to release a beta before it is ready. If you encounter bugs in your own testing or if you haven't run a thorough set of tests on your application, you have no business releasing into beta. When you release software that you know isn't ready for release, you are risking the trust of your users and the life of your product. While most beta users are willing to forgive minor bugs that pop up in beta, releasing a product that you know to be buggy is irresponsible.
When you release software that you know isn't ready for release, you are risking the trust of your users and the life of your product.
The second biggest mistake you could make in a beta release is to let in users too fast. While you can run stress tests, you won't 100% know what your server can handle until it's been put under a real test. If the server is going to crash, you want it to crash with 1,000 beta invites out and not 10,000 or 100,000 beta invites outstanding. Additionally, you are in beta, so it is possible that you will come across a bug or two. With Dunked, for example, we had an odd bug related to certain project slugs. Everything worked fine with the admin, but the live page resulted in a 403 error. It turned out to be a very simple fix. But because the issue was related the URLs, we had to correct existing URL slugs. It was certainly easier to search and remedy hundreds of project and page slugs than it would have been for hundreds of thousands.
One of the biggest limitations of a beta release is related to feedback. Granting users beta access, does not mean that they will provide you with feedback. Users may fully intend in provide feedback, but simply don't have the time put their feedback into words. They may also feel that they are doing something wrong, when the application doesn't work as they expect, and thus don't write you with feedback. Additionally, if you have to iterate over a certain component within your application, odds are that the quality and quantity of the feedback is going to wane.
I do not believe that it is not wise to try and include every single wanted feature when launching a product. I think you are better off launching a "completed" beta version, and then continuing to iterate over the product, adding additional features. In doing so, you'll be able to gather and use user feedback to improve your product.
Saying that, you must have a plan in place as to how you are going to monitor, gather, and act upon the feedback you collect. Thus, you'll be in a position to add features that create real value for the user, improving your chances of success.