This blog post is a repost of the original technical overview of the Trading Strategy protocol software stack for community feed aggregators. Please read the original blog post for the best layout and formatting.
The audience of this post is software developers who are looking to build scalable software-as-a-service solutions and are interested in Svelte, Python and TimescaleDB technologies. Developers who are interested in Web3, Ethereum, cryptocurrency and blockchain technologies will also find this post useful.
What are Trading Strategy protocol and algorithmic trading?
Trading Strategy is a new service for algorithmic and technical trading of cryptocurrencies on decentralised exchange (DEXes).
Algorithmic trading is a derivative of technical analysis; taking trading positions based on pure mathematics and data. Algorithmic trading is part of quantitative finance, the opposite of value investing where trading decisions are made based on fundamentals. Algorithmic trading provides a systematic approach to trading compared to methods based on trader intuition or instinct. Whereas technical analysis often aids humans to take trading positions, in its purest form in algorithmic trading a trading program follows a set of trading rules and independently executes trades on the market 24/7.
Today, Trading Strategy offers market data feeds for Uniswap compatible exchanges across three different blockchains (Ethereum, Binance Smart Chain, Polygon). We expect to cover all major decentralised exchanges and exchange types on all major blockchains by the end of the year.
Our mission is to make algorithmic by developing and investing in algorithmic trading strategies easy. Although serving the market data is the first step, Trading Strategy is going to grow beyond the information service website. During the course of 2022, the service will be decentralised, so that the whole software solution, strategy and trade execution will be runnable anyone independently (so-called operating a node).
You can read more about our future plans in our vision blog post.
Rich in technical wealth
Trading Strategy chose modern architecture and the best components of 2021 for its software development, making its codebase high in technical wealth.
Technical wealth is the opposite of technical debt. As Trading Strategy was able to start as a clean slate project, we had the privilege to pick the latest technologies and build our architecture around these. This enables us to build faster and do changes more dynamically. As an example, a lot of out of the box features offered by TimescaleDB would have taken us months if we had built them ourselves.
The decentralised finance (DeFi) industry itself is high in technical wealth compared to traditional finance (TradFi). This is due to the use of public ledger (open data), free-as-in-speech codebases (open source) and permissionless public blockchains (open access). This is the opposite of the traditional financial services that are built around closed networks, private APIs and exclusive access. Privileged and siloed software development leads to high maintenance cost long term, as there are no people who are able to work with the code. For references, see this blog post on legacy investment bank software and Google’s woes about maintaining their own Linux kernel with 9000+ in-house patches. Writing financial applications for DeFi is 10x – 100x more developer productive than writing them for TradFi.
What is a Web3?
From the software developer point of view, Web3 refers to backend-free services. No logins or registrations are needed. The user data is not processed and stored on a private server, but a public blockchain where the business logic is wired together using smart contracts. Any transaction, or a “POST request” in the traditional parlay, is initiated by users themselves, using their private key on their locally stored wallet application.
In this kind of model, there are no sysadmin fraud risks, no data loss risks or data liability risks with user identity information. The user is in 100% control. It is a fairer world where users have more control over the revenue streams they generate. Big IT like Facebook and Google have fewer unfair monetization opportunities on user data.
Only some services are a good fit for Web3. At the moment Web3 works best for pseudonymous and public data use cases. This is almost all finance, provenance (NFTs), public social media like Twitter and discussion forums and eCommerce.
Web3 is still early: though most pieces have been figured out in computer science theory, the implementation is still under development. For example, doing database-like traffic (ThreadDB) and storage (Arweave, Storj) can be still considered “alpha” today.
Software stack overview
Below is a walkthrough of how the HTTP requesting of Trading Strategy website is set up. Because Trading Strategy application is eventually going to be a Web3-like oracle node in a distributed network, everything built API first. There are no internal APIs – the website uses the same public market data APIs as everyone else.
Trading Strategy Oracle is a process that indexes blockchain data: DEX trading data, tokens and so on. Oracle and web processes communicate over TimescaleDB. Oracle is responsible for tasks like generating OHLCV candle data, generating liquidity maps and fetching US dollar reference prices. Oracle processes connect to various blockchain GoEthereum nodes that are part of the P2P network for the respective Ethereum Virtual Machine based blockchains.
Frontend: Why did we choose Svelte?
Svelte is a new JavaScript frontend framework and an alternative to React, Vue.js and Angular. Svelte comes with an integration package called SvelteKit that adds a standard web server, routing, server-side rendering and other core functionality needed to build a fully functional website.
We considered React for Trading Strategy . React is the de facto frontend framework choice of the cryptocurrency industry. However, even with our extensive experience in React, we chose Svelte because we believe Svelte is the framework of the future. Svelte offers reactivity with ahead-of-time compiled virtual DOM free approach. Svelte components are very maintanable single files, containing normal HTML template and normal CSS code instead of bastardized “styles-in-JS” or domain-specific templating approaches. Writing reactive logic in Svelte is easier than in other frameworks, due to its elegant design making the most developer-friendly frontend framework.
SvelteKit takes the developer experience of Svelte even further, by introducing the concepts of file-system based routing, simple server-side rendering and integrated web server (Vite). With SvelteKit’s batteries included approach an application developer spends less time on plumbing, boilerplate code and debugging async reactivity mess This all translates to better efficiency: the codebase is easier to read and maintain due to standardized coding conventions across open source libraries, components are faster to develop and new developers pick up the pace faster.
We also use components outside Svelte. We use uPlot for charting. Though the TradingView is the most popular JS framework for technical trading, we are puritans that go with 100% open-source approach as we are building for long term. uPlot was the charting library with an open-source license. For our number heavy tables we use DataTable library. For Web3 integration, we use web3.js through svelte-web3.
Backend: Why did we choose Python, Pyramid and SQLAlchemy?
Pyramid is a web framework for Python and SQLAlchemy is Python’s most popular ORM. We have experience writing Python applications since 2006 using Django, Flask and Pyramid. We also have experience writing backends in Node.js with various frameworks like Next.js.
For developers who know several programming languages, Python is superior choice for the backend. Writing Python programs takes fewer keystrokes. Python is the most readable of all programming languages. Optional typing support is a great way to make the code more static when the team grows.
There is a forever battle with sync vs. async. There is greater maintenance efficiency with colored function free, linear threaded, Python code. Our workloads are computationally sensitive, not IO sensitive, so using async IO would cause more headaches and we would get no benefits out of it.
We chose Pyramid and SQLAlchemy because we are doing API first backend and complex databasing with our special workload. Trading Strategy currently features tracking of 800k trading pairs (NASDAQ has only 3000). This is not your run of the mill CRUD and admin UI application. For this kind of use case, Pyramid and SQLALchemy are the choice today. This framework combo offers powerful tooling to cover advanced use cases without sacrificing developer productivity or being completely outside of an average backend developer skillset.
For our API tooling and developer communications we chose OpenAPI 3, also known as Swagger. Integration is mostly cost-free: pyramid_openapi3 validates the API definition, validates requests and replies, and can automatically route payloads to their corresponding endpoints and offers Swagger interactive API explorer. This all saves us a lot of manual development.
For background workers, we use Dramatiq library with Redis broker. Dramatiq task server is simpler, much easier to understand and maintain than more well-known Celery.
Data research: Why did we choose Jupyter notebook?
Most quantitative finance in the world uses Jupyter Notebooks and Python data science libraries like Pandas and Numpy. This is the strongest suite of Python that no other programming language can match. In fact, Pandas was originally developed by an investment bank.
Trading Strategy algorithm backtesting is done in Jupyter. We offer a Trading Strategy client library so that quants can develop their algos against decentralised exchanges without need to know low level blockchain specific details. If you are interested to see if you can beat the market please see Getting started tutorial (beta warning: better to join our Discord as well.)
Database: Why did we choose TimescaleDB?
TimescaleDB is a PostgreSQL extension specialised in time-series data, especially the “big” flavour of it. The alternatives included Clickhouse and xxx.
We chose TimescaleDB, because they are based on PostgreSQL. During the last 20 years, PostgreSQL has overtaken as the most advanced open-source database. PostgreSQL has the most vibrant database ecosystem on this planet. Tuning PostgreSQL is well-known: there are multiple products and companies to support in-house development.
Many of the core TimescaleDB features make our life easier. For example, hypertables provide fast insert times on large time-series datasets – otherwise inserting your one-billionth row starts to slow down. Continuous aggregates give us free upsampling 1-minute OHLCV candles to 5 minutes to 30 days periods.
TimescaleDB completed their Series B funding round where they raise $40M. Based on the quality of their product, the detail they go in their developer communications, we can see why they could be the winning horse of time-series databases.
TimescaleDB team has been the most responsive of any projects we have seen during the 25 years of open-source involvement. Their proactivity and helpfulness give us an assurance that TimescaleDB is serious about building an open-source community. All of our StackOveflow questions and Github issue reports, no matter how bad or novice has gone unanswered. We are pretty sure if one were to ask something offtopic, like making a coffee, and tagging with “timescaledb” it would still a receive perfect answer.
Why did we launch on Ethereum mainnet, Binance Smart Chain and Polygon?
When we started to build Trading Strategy early 2021, the layer 1 blockchains were not still on such rampage as they are today. Polygon and Binance Smart Chain were the layer 1 top dogs and no layer two was live yet. Ther chains had user adoption and active DEXes like PancakeSwap and QuickSwap. Today we have more competition with the likes of Avalanche, Fantom, Aurora and Telos EVM. Also we have non-EVM based solid solutions like NEAR, Elrond and Solana. We expect to integrate all of them during the source of 2022.
Ethereum mainnet has the best developer community. However, the Ethereum mainnet transaction costs are prohibitively expensive for the unforeseeable future, and thus it is unsuitable for our active trading strategies.
What challenges do we see?
Developing the means failing and retrying a few times, as your first guess is not always right one. We have some good lessons from 2021.
SvelteKit is still in beta. Some of the aspects of it are still under development and may not have complete documentation or supporting material like tutorial blog posts. We had to figure out a lot of aspects ourselves, especially what comes to SvelteKit server-side rendering speed and tuning. We feel the benefits of SvelteKit developer productivity greatly outweighs some learning curve and contributions to the documentation we had to do ourselves.
Svelte is new, thus it still does not have Svelte-native feature-rich charting libraries like uPlot and Datatables. They do not integrate to SvelteKit server-side rendering flow, making it not possible to serve pre-rendered pages.
Hosting Polygon and Binance Smart Chain nodes is tough. Both blockchain teams have issues with developer communications. There are no adequate manuals for running your own node. The expectation of using bug-ridden third-party API services goes against the blockchain ethos
Some queries on PostgreSQL are still unnecessary complex whereas other databases do better. Often we need to refer to the latest value, like the latest price, either for a single item or for a group. Unfortunately PostgreSQL does not offer any native “latest value” indices and it is often tricky to write an efficient query for this. Sometimes even simple ORDER LIMIT 1 seems to cause issues for PostgreSQL unless you create unnecessary fat indices.
We are open source
Trading Strategy frontend has been open source since the day one. If you are new to Svelte / SvelteKit you might find our repository interesting to read and learn why we have made certain design choices.
The same goes for the Jupyter Python trading strategy client. Oracle code and backend code will be eventually opened, as we start rolling out the protocol network later in 2022.
We are hiring
We are currently hiring for frontend (Svelte), backend (Python/PostgreSQL) and quant research (Jupytere Notebook/Pandas) positions. If you are interested to work with cryptocurrencies and algorithmic trading please email us careers@tradingstrategy.ai.
The cover photo by Alex Chumak.
Subscribe to RSS feed
Follow me on Twitter
Follow me on Facebook
Follow me Google+
from Planet Python
via read more
No comments:
Post a Comment