Nov / Dec '21 Update - Promising signs from our closed beta launch. Very promising indeed...

Go behind-the-scenes of our closed beta launch and see all of our learnings and feedback we've gotten from users so far!

a month ago   •   10 min read

By Tony Cai

Transparency is core to what we do at Atomic.Finance.

To that end, beginning this past June, we began to share publicly our monthly updates that we send to our investors and advisors. As always, if any of you have a new thought for us (it could be ideas, feedback), please give us a shout @AtomicFinance on Twitter! Thank you being on this journey with us to build #SoundFinance together. ๐Ÿ’ช

- Tony (CEO @ Atomic)


Email to Atomic Finance investors and advisors sent December 8, 2021

Recap from October:

In the update for October, we discussed how the largest focus was simply getting ready for the closed beta launch across the team - wrapping up the engineering and product dev work necessary. From the product / design side, the primary focus back in October was on preparing for the onboardings and solidifying the primary qualitative and quantitative learning goals for this beta period.

This monthโ€™s TLDR:

This past month and a half has probably been our most jam-packed month yet in terms of learnings and customer feedback. From:

  • launching our closed beta early November, and getting early feedback from customers on the "activation" flow in terms of getting started with the product,
  • to seeing an impressive number of users roll over from their first cycle to the next, and a significant number of users either increasing their BTC investment sizes from their first cycle to the next
  • to very encouraging qualitative feedback

From this, we saw some concrete early evidence that we're tackling a problem space people care about and our product is on the path to solving that problem.

That being said, it also wasn't quite completely smooth sailing either. During the process, we saw:

  • the more-frequent-than-we'd-like occurrence of small bugs in the app, primarily related to UI/UX. Moving forward, we're committed to improving some of our engineering and testing processes to reduce these occurrences.
  • our first downside protection being activated due to the Dec 4th capitulation. While the functionality did mostly work as intended and protected users from further losses, it did mean that just less than half of our users experienced a small loss on their BTC balance.
  • something we'll be looking out for this coming Friday is how many of these users who experienced downside protection will continue to utilize recurring strategies.

All in all, we could not have asked for more from a learning and validation standpoint in a month's time from an early closed beta. Overall, the user behaviour we've seen represents a significant improvement over the activity we observed with manual positions, where we didn't observe many users coming back and rolling over. This also gives us more confidence as a team to begin expanding our closed beta to more and more users on our way to a public launch next year.

Let's dive into the update!


๐Ÿ™‹๐Ÿปโ€โ™‚๏ธAsks:

๐Ÿšจ IMPORTANT: Are you connected with any crypto traders (doesn't have to be one looking for a job!)? Would you be open to making an intro?

  • We're looking to bring onboard a trader to help us part-time with backtesting moving forward to develop additional recurring strategies for different market conditions.
  • We're interested in opening conversations with different traders - even if they're not currently looking for a job - to expand our network in this arena and see if we can eventually get connected with a trader who would be a good fit.
  • So far, we've been enlisting the help of Joe, our options trading advisor to help us develop our first recurring strategy based on short puts. Moving forward, as his role focusses more on the market making entity, we'll likely need another person(s) dedicated to the backtesting.

๐Ÿšจ IMPORTANT: Do you know any CTO's or engineering leaders we might be able to enlist as mentors or have coffee chats with? (Especially those who have had experience scaling an engineering team)

  • Assuming things continue to go well with the beta, we'll be looking to scale up the engineering team next year.
  • Hence why weโ€™re looking to improving processes within our engineering team and learn from someone who has scaled an engineering team.

๐Ÿ“ˆMetrics

Note: Numbers DO NOT include activity from team / advisors

Recurring Strategies

  • Total value locked at peak: 5.7 BTC across 26 users
  • Currently 4.3 BTC locked across 14 users after 1.4 BTC of deployed BTC experienced downside protection
  • Average value locked per user: 0.23 BTC
  • Cumulative volumes: 12.117 BTC contracts and over 2 two-week cycles
  • Cumulative premiums earned by users: 0.128 BTC (US$6465)
  • (Key Activation KPI): No. of users onboarded to recurring strategies: 33 onboarded, with 26 of them having entered their first recurring strategy cycle: 79%
  • (Key Retention KPI): No. of users rolled over from first cycle to second cycle: 15/16 (94%)
  • (Key Retention KPI): No. of users increased investment amounts from first cycle to second cycle: 3/15 (20%)

Manual Positions

  • 7 current active positions across 4 users.
  • Total value locked: 1.5 BTC
  • Average value locked per user: 0.375 BTC

Waitlist Signups - Now at 7.5K (+200 growth)

  • Growth of the waitlist continues to be on the slower side. Likely a result of reduced word of mouth and shoutouts than usual (many of our existing users waiting for recurring to launch at the moment). Also, our social media activity was less frequent this past month as a result of heads down focus on product currently.

โš™๏ธ Engineering

  1. Squashing bugs: A major focus for this past month was fixing UI/UX bugs. Since we're dealing with people's money, we need to have a very high bar. Each bug erodes a bit of the user's confidence. Unfortunately, this meant significant dev resources spent on resolving issues rather than working on new features. This is one of the reasons we've decided to switching to one-cycle every 2 weeks to reduce time spent on bug fixes weekly. Additionally, we'll be spending the team to reduce technical debt in the app and re-architect how we handle state changes which should reduce the number of bugs we experience.
  2. Handling critical infrastructure issues: Throughout the live beta of recurring strategies, there were several critical pieces of infrastructure that failed, some due to bugs on our end, and others external.

a. Oracle attestment failure: On November 26th at 8 AM UTC, our oracle provider Suredbits failed to attest to the BTC price which is required to close contracts. They were able to resolve the issue several hours later, before a secondary issue arose. During the post-mortem we discussed the need to professionalize and improve the reliability of their systems, as it is critical for maintaining trust with our users. We are now in the process of developing an SLA with them.

b. Failure to activate downside protection: During the large Bitcoin dip in the early moring of Dec 4th, our infrastructure failed to activate downside protection. This was due to a prod-eng bug which failed to send a request to one of our other services. The bug has been resolved and we've added additional alerts for the team to be notified in case of critical infrastructure failure.

3. Continued market-making bot (kiwi-bot) development: We've continued development towards market-making bot functionality required for our market maker to hedge positions which enables us to profit from positions that users create.

4. QA + internal testing: For the past couple weeks we've been spending time building an app and infrastructure deployment checklist and internally testing the app to identify bugs/issues.

๐ŸŽจ Product / Design - Closed beta learnings

Early signs of product stickiness with Recurring Strategies

Earlier in the update I highlighted metrics arouund retention - whether users are rolling over and increasing their investment amounts:

  • While far from being conclusive - seeing a very high percentage of users roll over as well as seeing some users increase their investment amounts were good early indications of product stickiness.
  • This represents a significant improvement over the activity we observed with manual positions, where we didn't observe many users coming back and rolling over
  • This gives us more confidence as a team to begin expanding our closed beta to more and more users on our way to a public launch.

Areas of improvement - as identified via user qualitative feedback

For some users, better transparency around the details of each cycle

  • Things like the strike price of the cycle and cycle "health" - in terms of how close their strategy is to a downside protection trigger

Better wallet security

  • Something mentioned by quite a few users that are considering increasing their amounts. Currently the wallet within the Atomic.Finance app remains a single-signature hot wallet. Future improvements could include:
  • 2FA
  • Restricting sends from wallet to only whitelisted addresses
  • Multisig

Making it easier for folks to increase amounts ahead of Fridays

  • A few users mentioned that they were a bit pre-occupied on one of the Friday rollovers (Black Friday) and ended up not being able to increase their investment amount because they didn't have their BTC wallet on hand

Future improvements can include:

  • Showing an earlier prompt in-app or over email to encourage user to pre-deposit the increased amount rather than doing everything Friday.

Better prompts to remind users to authorize their first cycle

  • Some users just forgot.
  • We want to make sure we notify users reliably and via email. We can also add additional lifecycle email / notification in onboarding process

More education around strategies like what is a put and how downside protection works

  • A lot of our users are still very new to the world of options. So one of the things we're trying to improve on as we revamp the onboarding flow to be more self-serve is to include some basic in-app education about options and how they correlate to recurring strategies.

Learnings from our first downside protection trigger / stop loss.

  • During BTC's mini-capitulation event where BTC price dropped almost $10K in an hour down to $42K, we saw 12 out of our 26 users (and the 1.4 BTC they invested) experience their first downside protection trigger. This led to them losing 6.1% on their BTC balance invested in the recurring strategy.
  • The remaining 14 out of 26 users (4.3 BTC invested) did not experience downside protection due to the slightly different position they were entered into the week earlier as part of the recurring strategy.
  • From our backtesting, this was the first downside protection trigger to occur in the past year, and as fortune would have it - it happened just a month after we launched our closed beta.
  • While we were initially quite concerned about user reaction, especially with downside protection being triggered so early in many our user's journeys, we later saw this instead as an opportunity to learn from our users, and to learn what processes and product improvements we need to add moving forward
  • At the end of the day, downside protection being activated is indeed an inevitable outcome of the recurring strategy - and getting user feedback on this process earlier rather than later has been extremely helpful in identifying things to improve on for future occurrences of downside protection.

Our learnings:

  • One of the user criticisms following the downside protection being triggered was that we made misleading claims about how the downside protection functions.

Upon reflection, we agreed that this copy was incorrect and misleading.

  • Downside protection relies on stop losses to work and protect user capital from large losses in the event of an adverse price movement. While historically, downside protection has limited losses to 4.8 - 5% each 2 week cycle, the reality is that the size of the loss can vary slightly based on available liquidity and market volatility.
  • The downside protection this time around is a good example of this: leading to a 6.1% loss.

We've since updated this copy in-app and we're working on getting user feedback on the updated copy - does that change their perception of the strategy and the amount they're comfortable deploying?

  • As a result of this misleading copy, we decided to compensate users who experienced this downside protection for the difference between 5% they were expecting and the 6.1% they actually experienced

A few additional screenshots of user feedback!


๐Ÿ“– Content / Social / Community

While still small, our community of 50 or so has been going strong and maintained a consistently high level of activity since our closed beta launch

  • Only onboarded users were permitted to join the Discord community
  • We've seen numerous users be very proactive in alerting us to issues and providing detailed feature requests along with their reasons they want that feature
  • It's feeling like we've been able to build a solid foundation of a larger community moving forward

๐Ÿ”ฎ So what next?

Our primary focus in the next month:

Product and Design:

  • Self-serve onboarding flow and in-app education as we begin the scaling process and detach ourselves from needing to be on every onboarding call (Streamline user's initial activation path as much as possible)
  • Small UI/UX enhancements following qualitative feedback from customers (eg. enabling visibility and transparency into
  • Decide on whether to discontinue manual positions
  • FAQ: We've collected tons of FAQ's that users have been asking us over the past 4 weeks! Since we're big on transparency, we've set aside some time over the next few weeks to write some of them up.

Engineering:

  • Improving testing and reliability: Increasing test coverage on all projects and reducing technical debt in the app and several of our services
  • Increasing documentation of various projects and network architecture to allow for external review
  • Continued development and testing of market-making bot
  • Additional logging, alerts as well as setting up an on-call schedule for the engineering team during critical events

Growth:

  • Tony to start fleshing out a plan for growth and expanding the beta user base.

Hiring:

  • Hope to finalize trader hire by January so we can continue to improve on the existing strategy and conduct research on additional strategies for different market conditions (eg. a hedging strategy for a bear market - if that is to come)
  • Following trader hire, we're thinking about a further engineering hire next year.

Spread the word

Keep reading