- Categories
- Getting Started
- Technical
- Understanding Your Results
- Why StoreMaven?
How much does it cost to test with StoreMaven?
We tailor our services to each enterprise customer and their unique app store marketing needs. Contact us for more information, and we’ll be in touch shortly: sales@storemaven.com.
Why don't we show your Confidence Level right away?
Before closing a test or closing down any variations, we want to ensure that the test results are stable and not likely to shift dramatically.
When a variation has less than 200 users, as well as less than 30 installs, just a couple of users can dramatically shift the performance results, as each individual user holds more weight than when the sample size is larger. Before those thresholds are met, it is likely that you would see high confidence one day and a dramatic decline the next.
We know in the initial days of a test, page performance can be more volatile due to the quality of users who are initially sent in through your traffic campaign.
As such, we require 3 full days of traffic before we can start recommending any action.
If you have any further questions, please feel free to contact us at support@storemaven.com
What are the benefits of predictive testing?
Our proprietary app store predictive algorithm, StoreIQ™, based on a well-known algorithm called Multi-Armed Bandit, is designed to save you between 30-50% of the cost of each test by concluding tests faster and with fewer samples. All StoreMaven tests utilize StoreIQ™, saving you precious budget and time.
What are the advantages over traditional A/B testing?
Predictive testing allows StoreMaven to quickly identify and close low performing variations, as well as assess a test’s significance faster and more accurately. Traditional A/B testing requires sending an equal amount of traffic to all variations until the end of the test, resulting in a higher sample size and a longer test time.
Can I test multivariate pages or elements?
Multi-Armed Bandit experiments (which StoreIQ™ is based on) are optimal for multivariate testing environments, identifying winning combinations, closing weak variations early, and taking into account additional engagement metrics. Predictive testing similarly benefits two variation tests.
What professional services does StoreMaven offer?
StoreMaven’s dedicated client services and analytics teams provide support and insights from the pre-testing stage through the post-test period. Leverage our expertise in data analysis to get tips and learn best practices, monitor live tests and receive detailed reports with industry benchmarking and next testing steps.
How can I contact StoreMaven's support team?
How can I change my password?
To change your password, click the “Forgot my password” link in the login window. This basic manual will guide you step-by-step.
Who is The Maven?
The Maven is your Artificial Intelligence-based success manager. The Maven will analyze test results and test status to alert you via email or notification when actions are required in a given test, such as closing a variation or a test or that your test has started to receive traffic.
The Maven is not available in StoreMaven Lite.
What metrics do StoreMaven tests measure?
Your Test Lab dashboards provide you with an overview of your test as well as look at more specific metrics. In the Overview Dashboard, you will see the overall significance of your test as well as CVR by day for each variation and as a cross test.
Other dashboards include traffic and device, users (funnel). engagement metrics, screenshots and video. The funnel looks at your users’ behavior, while the engagement dashboard lets you view gallery scrolls and clicks, “read more” and reviews. You can also look at impressions and clicks per image and per variation, as well as video “play” clicks, video completion rates, install rates, watch time and watch rate.
StoreMaven also provides basic and advanced analyses, as well as custom reports. Contact your StoreMaven account manager to learn how to request these services.
When should I close a test?
From the overview dashboard in StoreMaven’s Test Lab, it is possible to see the test significance level at any given moment. We typically recommend closing a test once it has reached a significance level of 95%. In addition, The Maven will alert you when your test can be closed.
You can also turn ON Autopilot from the overview dashboard to automatically close weak variations saving time and money.
Feel free to reach out to your StoreMaven account manager should you have any questions regarding when you should close your test.
How many impressions per app page are needed to conclude a test with significant results?
Depending on traffic source, impressions can number from hundreds to tens of thousands. This is determined by a number of factors. First of all, the more variations you have per test, the more impressions you will need to reach significance.
In addition, the conversion rate per variation will also determine how many impressions you will need. The bigger the gap in CVR between your leading variation and other variations, the fewer impressions you will need.
We typically recommend accounting for 1000 impressions per variation to begin a test. Once your test has received enough traffic, it is possible to estimate how many more impressions you will need in order to conclude the test.
How is significance calculated?
We calculate significance based on the Multi-Armed Bandit algorithm. Significance is calculated three times a day and is visible in the Overview Dashboard in StoreMaven’s Test Lab.
What are StoreMaven’s professional services?
With our custom plans, StoreMaven offers app store benchmarking by industry and advanced analytical support, as well as a dedicated business intelligence and optimization analyst.
In addition, all StoreMaven plans offer premium customer support and a personal account manager. It is also possible to request advanced analyses as well as customized reports.
Contact the StoreMaven team at support@storemaven.com for assistance in basic analyses or questions.
What can I test with StoreMaven?
It is possible to test the following store properties with StoreMaven:
- Icon
- Title
- Rating & Votes
- Screenshots
- Video
- Poster Frame (iOS)
- Feature Graphic (Google Play)
- Description
- Uber Page (iOS)
What creatives do you need to start a test?
In order to start a test, you need to have the same creatives that are currently featured in the app store that you are testing.
In iOS, you will need an icon, screenshots and video, if applicable. In Google Play, you will need a featured graphic, icon, screenshots and video, if applicable.
Can StoreMaven help me setup my test?
Yes! StoreMaven is proud of its great support team!
Feel free to contact us to arrange an initial training session or regarding any specific questions at support@www.storemaven.com.
How do I create a test in StoreMaven?
Enter your app name or store URL to pull your creatives straight from iTunes or Google Play to our Design Studio or upload them yourself for pre-launch apps.
Once your page is set up, duplicate your page and edit the page variations. We recommend creating 3-4 variations and testing one variable at a time.
Once you’ve created your variations, activate your test to generate a test link to get the ball rolling.
Can I run tests for both Apple’s App Store & Google Play?
You can! StoreMaven is fully compatible with both iOS and Android, so you can design and test assets on both platforms.
In addition, one of the benefits of StoreMaven’s platform is that it is possible to test assets that aren’t compatible with a specific app store before making costly updates. For example, prior to creating a special video for iOS that meets Apple’s requirements, run a video iOS test with StoreMaven to see how your video performs to determine what aspects you should keep or change.
Can I test in different languages?
Yes! StoreMaven currently supports testing in 16 different languages (German, English, Spanish, French, Hindi, Indonesian, Italian, Japanese, Korean, Dutch, Portuguese, Russian, Thai, Turkish, Chinese - Traditional and Chinese.)
Select your test language under advanced settings when creating a test.
Adding a language is easy and can be done upon request. Send us an email at support@www.storemaven.com and we’ll get on it.
What traffic should I send to my StoreMaven test? Does it matter?
It is possible to send traffic from any ad source such as Facebook, Instagram, AdWords, etc., however, the type of traffic you send will influence the results of your test.
The more your campaigns match your actual campaigns, the more reliable traffic is.
We have found that Facebook mobile ads most accurately mimic organic traffic. However, only you know what is the best type of traffic for your test.
Can StoreMaven drive traffic to the tests for me?
StoreMaven does not drive traffic to your test.
You know your audience best, so you know who best to target. However, feel free to contact your account manager regarding any questions or help in driving traffic to your test!
How can I run Facebook ads for StoreMaven tests?
Facebook’s Power Editor enables you to create “Clicks to Website” or “Website to Conversion” ads, as well as the option to customize your call to action. The best practice is to duplicate existing ads that work.
Can I track install and post-install behavior during StoreMaven tests?
Once you’ve activated your test, there is a field to add a unique attribution URL from the attribution service of your choice such as AppsFlyer, Tune, Adjust, Kochava or others. You can choose to create an attribute URL for the entire test or you can create a unique URL for each test App Page.
How much technical expertise is required to test with StoreMaven? Does it require SDK integration?
Using StoreMaven does not require any integration or SDK, so no level of technical expertise is required.
You will be able to import creatives either from the app stores or directly from your computer. All you need is an active internet connection!
How much does it cost to run tests?
In addition to your StoreMaven plan, you will also need to pay for the cost of sending traffic to tests. The cost will depend on your conversion rate, the number of variations you test, your cost per install, and the differences between the variations. To learn more about the cost of traffic and StoreMaven's pricing and packaging, please contact sales@storemaven.com
What kind of increase in CVR can I expect from a StoreMaven test?
App Store Testing can increase your app’s conversion rate by up to 50%. This begins with the first impression, the initial view a user will receive of your app store page. The layout of Apple’s App Store and Google Play differ so what may work for one won’t necessarily work for the other.
In iOS, the most important elements are the icon and screenshots, while in Google Play, the short description and featured graphic play a significant role. So, for example, a better version of your first two screenshots in iOS will increase CVR by 25%.
Surprisingly, only about half of your users will choose to interact with other elements on your page beyond the first impression such as scrolling your image gallery or reading the description. However, these users will be higher quality users as they will know more about your app, thus decreasing the bounce rate after installation. Get better quality conversions by improving content for these users. Adding an App Preview Video can increase installs by 23%, while a more engaging set of screenshots can increase CVR by more than 35%.
How much by which you can expect to increase your conversion rate is dependent on the quality of your marketing content. Check out this deck to for tips on how to make the right improvements to your app store page.
How do I know what my install conversion (CVR) is today?
You have a handful of different options to find out what your conversion is from ad click to download.
Among them are:
- App store analytics on iTunes Connect and the Google Developer Console
- Attribution services such as AppsFlyer, Tune, Adjust and Kochava
- Your conversion rate from click is calculated as follows: Ad Click / Installs from Click
- Your conversion rate from app page views is: App Page Views / App Page Installs.
Will I get installs from users participating in a StoreMaven test?
Users who click on the “Get” button from a StoreMaven test site are redirected to the app store where they can install the app. It is also possible to redirect users to a different landing page, survey or website for reasons such as testing a pre-launch app.
Some users will drop after clicking on the “Get” button and before they reach the relevant app store page. The potential drop from a StoreMaven landing page to installing on the app store depends on the difference between the test page creatives and the current creatives in the app store.
However, the potential CVR increase and insights derived from StoreMaven tests more than make up for any potential drop during testing.
How reliable is the StoreMaven platform?
In short, extremely reliable. There is no difference in behavior between how users interact with an app store and how users interact with a StoreMaven test. This is because users are unable to detect the differences between a test page and the actual app store.
The StoreMaven platform has been tested by tens of millions of users and is capable of operating at any required scale. Our predictive testing algorithms are able to identify winning variations and calculate significance, which allow you to confidently make changes to your live app store page.
Which App Stores are supported in StoreMaven?
StoreMaven currently supports Apple’s App Store and the Google Play Store
What is StoreMaven’s Design Studio and Test Lab?
The Design Studio and Test Lab are part of StoreMaven’s self serve platform. Use the Design Studio as a standalone tool to create new app store pages. You can design, preview and share your various pages all from one place. You can also compare up to four different variations side-by-side.
The Test Lab is where you will see results from your StoreMaven tests. Get an overview of your test and analyze different metrics using our dashboards.
How does StoreMaven work?
StoreMaven creates cloned versions of your app store page as mobile optimized landing pages. These pages includes all of your app store assets (Icon, Screenshots, etc.) and look just like a standard app store page. You can choose to send users to your app’s mobile app store pages from any mobile placement - Facebook Ads, Instagram Ads, AdWords, etc.
StoreMaven tracks all actions users perform on the page. The data is then uploaded to your StoreMaven dashboard where you can see which app store variation performed the best and a ton of metrics regarding your app store’s engagement.
Is there a demo account I can use?
Definitely! Once you request a demo, you will be able to enter and browse StoreMaven. This will allow you to get a feel for the capabilities of our platform and studio. When you are ready to request a test you can email sales@storemaven.com for assistance.
What devices are compatible with StoreMaven tests?
StoreMaven is compatible with all iOS devices and all Android smartphones.