Last night ShopSavvy 4.2.1 went live in the AppStore for iOS. I would argue this is a major update despite the fact that the developers called it a minor update. First, we uncovered a misalignment in focus that slowed down our scanner. By realigning the focal point ShopSavvy is now able to scan some of those hard to scan products much faster now. Second, we simplified navigation by keeping the toolbar on each screen. Now you can push the scan button from any screen in the app – this has a side effect of making the app more stable. Third, we ended the iAd experiment – no more iAds in ShopSavvy. Finally, we consolidated all product ‘options’ into a single button. After you scan a product you can interact with it using the option button in the upper right corner of the app. Options include: a) Edit Name & Image, b) Add Local Price, c) Share This Product and d) Save to List. Each of these options were previously available, but many users had a hard time finding them.

Take a minute to update to the new version. Remember that you need to update your rating so that it will show up in the ‘ratings for this version’ section. We want to see if we can keep this version at 5 stars. I personally think the work the team did definitely deserves 5 stars…

UPDATE 2: Okay, I’ve got another update. First, there are some of you who haven’t seen the app crash on your Thunderbolt (we have no idea how this could be the case), but for most of you (like us) ShopSavvy crashes hard almost immediately. Our lead Android developer explains what is going on in this cut and paste job:

After much toying, tweaking, and hacking I can safely say that the problem we are encountering with the HTC Thunderbolt can not be fixed from our side of the fence.  Doing testing across the Nexus S, Nexus One, Evo, and Thunderbolt I have pinned the problem down to the garbage collection mechanisms on the Thunderbolt; both on the native heap and in the dalvik.  On each product screen we place a large image of the product on the lower half of the screen.  Each time we create this image an allocation is done on the native heap to hold the actual Bitmap bytes while a small reference object is placed on the dalvik. The error that currently causes ShopSavvy to crash involves the native heap growing until the operating system can no longer allocate anymore space causing the program to crash from the native side of the application, not the dalvik.

In order to free memory on the native heap an Android programmer can either ensure that 1. The bitmap reference on the dalvik is garbage collected which will in turn call the native destructor which will delete the buffer on the native heap or 2. Call the recycle() method on the dalvik Bitmap object which will immediately delete the buffer on the native heap.  By calling delete on the native buffer the operating system should then mark that buffer space as reusable and the next allocation should be able to take advantage of the space ensuring that the native heap does not need to grow in size.  Therefore, to track down the problem I needed to ensure that both methods were used to keep the native heap under control.

The first round of tests involved ensuring that the Bitmap.recycle() method was called on each product image to ensure that the bitmap should then be immediately recycled.  I included a Bitmap.recycle() and a logging statement on activity shutdown methods as well as in the activity finalizer to ensure that the method was called and the bitmap destroyed.  On the Thunderbolt calling recycle() on the bitmap never caused an increase in free native heap space.  Therefore every time the product screen was visited another 1-2mb would be allocated onto the native heap until the program would get an OOM (Out Of Memory error).  If we compare memory consumption of the Evo with the Thunderbolt; after viewing 6 product screens, the native memory usage on the Thunderbolt would grow from 6Mb to 24Mb, where the Evo would grow from 9.9Mb to 14Mb (where it stayed).

The next round of tests determined the activity of the Dalvik garbage collection mechanisms.  I went through the code and put printouts in the finalize methods of our product screen objects. This ensures that I get a message every time the dalvik deleted one of these objects from memory. I also wrapped our BitmapDrawables to ensure that the finalize method was being called on them as well.  On the Nexus S, Nexus One, and Evo the garbage collector would kick in fairly quickly; after about 4 product screen views.  I received messages that the activities, tasks, drawables, and tasks associated with the product screen were all purged from memory.  This was also confirmed by my native heap gaining free space and not growing in overall size.  The thunderbolt however never called the finalize methods on anything other than Tasks.  This means that every activity was still being kept in memory along with WebViews and BitmapDrawables. Even though the references to all objects were deleted on the activity shutdown methods; the objects were never cleared from memory causing large memory usage.

I’m unable to recommend any fixes at this time.  All Android best practices are being exercised in our app.  We have concluded that all references are being deleted and all bitmaps have been recycled either by manually calling recycle or through the Bitmap object being finalized.

UPDATE: The bug is NOT network IO related, but bitmap related. Thunderbolt’s version of Android is allocating WAY too much memory when we are trying to create a bitmap. We are still stumped. What if we delivered a TIFF instead? Maybe a PNG? Just kidding – working on it now.

If you are a proud owner of the HTC Thunderbolt you will note that MANY Android apps seem to be crashing. ShopSavvy will crash almost immediately. We know why it is crashing, but we aren’t sure how to fix it. Here is a lay person’s explanation of what is going on:

When trying to create an image that should be 3k the system is allocating a 16mb buffer to load the bitmap. As you can see below the bitmap size is 16mb when it should be like 3k.

E/AndroidRuntime(11212): Caused by: java.lang.OutOfMemoryError: bitmap size exceeds VM budget(Heap Size=8239KB, Allocated=6688KB, Bitmap Size=16371KB)

Andy Rifken points out that HTC seems to have modified the Apache HTTP client libraries. His issue is with the default socket buffer for HttpParameters API, but we think that if he is right and they did modify Apache on the device it may be the cause of the problem we are facing. There is a work around Apache that we are currently working on – if it works look for an update shortly. If not, we will be forced to wait on a response from HTC.

ReadWriteWeb has a post titled, “2011: The Year the Check-in Dead” that basically argues that check-ins for check-ins sake are dead. I would tend to agree. There was a time a year ago when check-in apps were hot and we began to talk about ShopSavvy as a retail check-in application. In essence each time a consumer scans a product they are also ‘checking in’ at a retailer’s location. ShopSavvy determines store location after each scan to be able to offer local prices from nearby retailers as well as special location specific deals and coupons. This check-in is transparent to the user and does not require him to remember to check-in – ShopSavvy does it for him so that he can get the best possible deal. If you look at our unique check-ins last quarter the number is approximately 23 million (7+ million per month). At the end of the day we believe check-ins can be VERY valuable to both the consumer and the retailer, but checking in isn’t a natural act. Consumers rarely remember to check-in while in retail. If you can connect their check-in to an activity – i.e. like scanning a barcode to find the best deal – we think you can make a positive impact.

ShopSavvy, the leading mobile shopping assistant, announced today that nearly 12 percent of the 69.5 million smartphone owners in the United States are current unique users of ShopSavvy.

ShopSavvy today released usage data including total downloads, total updates and current unique users. The company reported:

- 16,000,000 downloads/installs. This is the total number of times ShopSavvy has been downloaded from the app markets of Apple, Android, Amazon and Windows Phone 7, as well as from independent markets or directly from the APK application file. This number does not include pre-loads or app bundles. It also does not include update downloads.

- 64,000,000 updates. This is the number of times that ShopSavvy users have downloaded updates to the most recent version of the app.

- 8,000,000 current unique users. This is the number of people who have used the app and currently have it on their phones. The current unique user number is lower than the download number for a variety of reasons. The biggest single factor is that when Android users do factory resets on their phones and then download the app, it registers as a new install. The current user number also does not include individuals who have downloaded the app but not used it, or who do not currently have the app on their phones.

“Usage numbers can be confusing. They are difficult to track in some cases, and different app makers define their terms differently. Many, for example, cite total downloads as their user number, when this number invariably includes duplicate users, as well as individuals who have downloaded the app but haven’t used it,” said Alexander Muse, ShopSavvy co-founder and CEO. “We decided to release more detailed usage data, together with our definition of terms, to present a clearer picture of the market penetration we’ve been able to achieve.”

ShopSavvy intends to update user statistics each quarter, Muse said.

69.5 million people in the United States owned smartphones during the three months ending in February 2011, according to comScore MobiLens.

About ShopSavvy

ShopSavvy is the leading mobile shopping assistant, with 16 million downloads and 8 million active users. ShopSavvy empowers smartphone users to quickly and seamlessly locate, research and buy products at the point of sale. When users scan a barcode or input a product name, they discover where the item is sold locally and online, where it’s in stock and at what price. ShopSavvy aggregates product data, deals, ratings and reviews from retailers, partners and its own users to provide the most comprehensive source of information and advice for mobile shoppers anywhere. For advertisers, ShopSavvy delivers “hyper-relevant” deals, promotions, warranty offers, and other UPC/GPS-targeted content to consumers. For more information, visit http://shopsavvy.mobi.

Ever wonder what it takes to rank in the top No. 25 Overall or No. 1 in the Tools category in the Windows Phone 7 App Market? ShopSavvy is currently No. 1 in Tools and No. 22 Overall (we have been as high as 14 overall). We are even beating Google! Here are the particulars:

- 800 downloads per day on average (top day was 3,172)
- 72,881 cumulative downloads

The WP7 version of the app is handicapped by Microsoft’s camera policies, but we have high hopes they will open up the camera to developers soon. Here is a screenshot showing our downloads:

The team uploaded the latest iOS version of ShopSavvy to the App Store. The update resolves a few long standing issues with the 4.0 version of ShopSavvy.

First, lots of users didn’t realize they could scan QR codes with the app. We decided to show the user what sort of codes he could scan on the scan screen. Now there is a picture of a UPC as well as a QR Code. The user can tap them to learn more about each type of code or they can simply scan the codes. Additionally, we have retained the popular ‘Shake to Scan’ screen. Finally, we determined our focal point was not in the center of the screen – for some reason it was just under the spinning ‘scanning barcode’ symbol. We moved the focal point to the center of the screen just under the red line.

Second, the tool bar turned into a tab bar whenever a user looked at a product screen. This always bothered me since I couldn’t figure out how to get back to the scan screen or how to use the List or Deals feature. Very few users figured out the three lines at the center of the tab bar actually let you slide the screen up to see the scan screen. It was sort of a cool feature, but it was buggy and caused the app to crash in other areas. Additionally, we had three different new buttons that were confusing. In the lower left part of the screen you could save the product to a list. In the lower right part of the screen you could send the product to a friend. In the upper right corner you could edit the product. In the updated version there is our standard tab bar at the bottom of the app and an “Options” button in the upper right corner. Tapping “Options” allows the user to A.) Edit Name & Image, B.) Add Local Price, C.) Share This Product or D.) Save to a List. We think these changes will make ShopSavvy more stable and easier to use.

Third, we added iAds to the retailer listing screens and besides not generating much revenue they screwed up the UI of the app. They slid content down and then when you moved forward or back through the app the area vacated by the iAd was empty. We could have resolved this through a bug fix, but we decided to pull the iAds altogether. No more iAds. Here is a picture of the new screens:

When we released ShopSavvy in 2008 (almost three years ago, picture from that time to the right) we had no idea how big it would become. I wouldn’t have bet on our success since we had never built a mobile app nor did we have any retail shopping experience (i.e. we aren’t shoppers). Perhaps in spite of ourselves we created something that really resonates with users. In 2008 very few consumers even contemplated using a smartphone to help them while they were shopping in local retail. But in 2009 studies were showing a small, but growing number of consumers (approximately 15%) were pulling out their smartphones to use apps like ShopSavvy to help them make buying decisions. IDC Retail Insights claims this number grew to 45% in 2010 and 2011 looks to be a banner year for shopping assistants like ShopSavvy. So how big are we?

We are actively releasing our numbers to a number of folks in the Bay Area and as a result there is a lot of confusion about how many users we have. We thought it would make sense to simply reveal the numbers to everyone.

- 16,000,000+ Downloads/Installs (200-275K a week)
- 64,000,000+ Updates
- 8,000,000+ Current Unique Users
- 3,100,000+ Actives in the last period
_____________
= More than 10% of smartphone owners use ShopSavvy

Downloads/Installs refers to people who have downloaded our app from Apple’s App Market, Android’s App Market, Amazon’s App Market, Windows Phone 7 App Market, directly from an APK and from one of more than a hundred independent markets (all Android). We don’t count pre-loads or app bundles from folks like Sprint and Handango.

Updates refers to people who have downloaded the app AND actually updated it when prompted by the market or the app.

Current Unique Users refers to people who are using the app and have it on their phones. This number is about half of the downloads for a few reasons. The biggest reason for a big download number compared to Current Unique Users is that Android users keep updating their phones and doing factory resets racking up the the download numbers.

Active Users refers to people who have actually scanned a barcode or used the app during a specific time period. In our case our active users is lower than uniques because not everyone shops every day, every week or every month. In fact half of our scans are consumer electronics which consumers shop for on more of a quarterly basis instead of a monthly basis.

Some apps call downloads users and we used to do so as well. We use the “more than” phrase a lot. With download rates between 200,000 and 275,000 per week it is hard to give a firm number. We have decided to focus on quarterly numbers – i.e. revamping our numbers each quarter instead of guessing week by week. I hope this post helps clarify.

The Android team at ShopSavvy quietly released an update to the Android version of ShopSavvy last week without much fanfare. Here is what is new:

- Complete overhaul of ‘DEALS’ screen. ShopSavvy now shows local deals from retailers near you.
- Users can turn on the flash to scan barcodes if they like. I wrote downside of this here.
- App gives you a button to turn on local if you have disabled it for some reason
- Several annoying bugs have been fixed

The early results from Flurry are positive. We are seeing a DEALS click through rate of 42% up from 9% before the update. User comments include:

- “Finds every deal possible and it saved me a lot of time and money!”
- “Coolest app ever! Upsetting to find out how much I’m overpaying on things!”
- “Perfect app!”
- “Way better than the other apps”
- “Brilliant app fast and reliable”

Three days ago I wrote a post describing a report that claimed Android’s marketshare was greater than RIM and Apple. Not a single Blackberry user complained, but hundreds of iOS users took the time to write to me or comment. They were mad. Based on their feedback it is clear I gave them the impression I was anti-Apple. I wanted to set the record straight that nothing could be further from the truth. Personally I use an iPhone, iPod, iPad and MacBook – my wife has an Air and an iPad and my son has an iMac and an iTouch. We are most definitely an Apple family. But I don’t consider myself an ‘Apple fanboy’. I am frustrated by the AppMarket sometimes, I am bothered by some of the decisions Apple makes and I am not afraid to voice my concerns. When I meet someone who can’t admit that Apple can make a mistake I know that I am in the presence of an Apple Fanboy. Of course when it was time to hire someone to ‘own’ the iOS version of ShopSavvy we decided it made a LOT of sense to hire one of these so called fanboys. Jake is UBER-passionate about Apple and their products.

Of course our experience with iOS hasn’t been perfect. When we launched on Android we developed ShopSavvy entirely in-house, but in 2009 we released an iOS version of the app we didn’t have any Apple talent on the team. As a result we farmed the development out to a group in India and after repeated delays had an app ready to release right before Christmas. The app sucked. It didn’t scan well. It crashed. It was generally a mess. Despite this reality more than a million users downloaded the app because our shopping data was far superior than our competitors (who admittedly had a much better scanner/app). The reviews were horrible – mostly one star ratings. I felt sick to my stomach and wasn’t much fun to be around that Christmas.

When we got back after the holidays we decided it was time to bring iOS development in-house and completely start over. We recruited a very talented iOS developer (i.e. that fanboy) who was interested in building something amazing. It took a little longer than I had hoped, but by November we had released a completely new app with a new UI, a new scanner and several new features. The new version rocked. Users really liked it. Called ShopSavvy 4, each 4.X version has been rated four stars or better (compared with an average of 2 stars for the original version).

Ironically, our Android users were REALLY frustrated we were spending so much time building an amazing iOS app. Of course they were right, we were spending most of our time trying to get the iOS version of the app right and as a result we didn’t release ShopSavvy 4 for Android until January.

Matt Weathers joined our team from ebay and immediately set about trying to determine how we stacked up to the competition on iOS – namely to Red Laser (owned by ebay). He hired a research firm to follow smartphone shoppers to watch them use both apps to determine which they preferred and why. The vast majority of users, after using both in a shopping setting, preferred the look and feel of ShopSavvy as well as the superior product results. The numbers support his finding as ShopSavvy 4 on iOS has grown downloads by more than 250%.

More than 7.5 million smartphone owners use ShopSavvy to scan barcodes when they shop. With each scan our users ‘check-in’ at a retail location giving us valuable location data so that we can deliver them relevant deals to help them save money. Some of our users are adding product data including product pictures (more than 2 pictures a minute) and product pricing (more than 10 additions a minute). Each month our users scan millions of barcodes requiring us to deliver a LOT of information (product info, product reviews, local prices/inventory and online prices) back to them – billions of pieces of data in the course of a month. During the course of the year it takes about 20 servers (not including development servers) to keep everything running.

- 3 PriceNark 4 API servers
- 2 PriceNark 3.2 servers
- 1 Image server
- 1 Rails box (running AdOns and misc API endpoints)
- 1 Tomcat box (running misc backend API endpoints)
- 1 Database (MySQL) server
- 1 Database (MySQL read replica) server
- 3 Chalupa servers (barcode decoding)
- 6 Cluster servers (Hadoop/Cassandra/HDFS)
- 1 WordPress (shopsavvy.mobi)

During new releases or during special deals we sometimes need to spin up new instances of PriceNark and Chalupa. During Christmas we double all of these server counts to deal with the increased shopping traffic. We also leverage Amazon’s cloud for redundancy – we can run completely in the EC2 cloud or on our own servers.