img

Run Your Own Crypto Trading Bot on Google GCP Cloud Functions

img
valuezone 10 February 2023

Run Your Own Crypto Trading Bot on Google GCP Cloud Functions

Tired of buying your cryptos too high and then selling too low? It doesn’t have to be that way! Here is a fun project for building your own trading bot. This article uses Python as it is the easiest programming language to learn (my personal opinion).


Foto von Wance Paleri auf Unsplash

I will show you how to create a trading bot in Python that will automatically buy and sell crypto currencies via the Coinbase Cloud API based on certain trading indicators. From a technical perspective, the trading bot will be deployed on Google Cloud Platform (GCP). It uses Google Firestore to keep track of all initiated orders. The bot runs as a scheduled cloud function. Here are the things you will learn in this article:

  • You write Python code that connects to Coinbase Cloud to retrieve the necessary crypto data to buy and sell crypto using your personal Coinbase API key and secret.
  • We enrich the available Coinbase data with custom trading indicators (Bollinger Bands) and we will also pull in trading recommendations from the Trading View API.
  • Once a buy and sell order is created, we will keep track of this information in our own Google Firestore database.
  • We will deploy the Python code as a Google Cloud Function on GCP and we will schedule the function to execute every five minutes via the Google Cloud Scheduler functionality.

This article focuses on the conceptual setup with a strong focus on the overall mechanics. The full working code including a step-by-step deployment instruction is referenced at the end of the article. Feel free to play around with the code and deploy it in your own Google GCP project.

Trading Bot Strategy

The trading strategy of the bot suggested in this article is pretty straightforward but offers enough on-ramps so that you can build your own algorithm on top of it. The bot will run every five minutes to check if the following three conditions are true:

  • The Trading View API returns a one minute trading recommendation of either BUY or STRONG BUY for the chosen crypto currency.
  • The last buy order is longer than 24 hours ago (to avoid too many frequent trades in cases the price keeps falling and falling).
  • The current price of the crypto currency is below the 20 day simple moving average (SMA) based lower Bollinger Band indicator. To find out more about trading indicators, visit the following article.

If all three conditions are met (meaning there is a good dip in the price with a positive market signal), the bot will trigger a market buy trade. Once a market trade is executed, the bot will automatically create a stop limit sell order with the goal to achieve a certain profit margin. Of course, the algorithm relies on rising market prices in the long term.

Prerequisites

First, you need your own account with Google Cloud Platform and create a dedicated project for this article. Sign-up with GCP should be fairly easy and you probably want to take a basic tour through their platform to better understand the components we will use. It is also recommended to take their Cloud Leader certification if you have some time. We will use the following GCP features for this article: Cloud Functions, IAM, Security, Cloud Scheduler, Firestore. You also need to install the gcloud CLI on your local machine to conveniently manage your Google resources.

Second, you must have an account with Coinbase. In Coinbase, you must create your own API key and secret under Settings/API. Coinbase delays the activation of your API key and secret by 48 hours as of writing this article. Needless to say, that you also need to transfer some money to Coinbase so that the trading bot has the resources to trade cryptos.

Google Cloud GCP Architecture

On your local machine, you need a project folder. All Python code is in one single main.py file. Your external pip libraries are listed in requirements.txt. Environment variables are kept in file .env (only needed for local testing) and an .env.yaml (uploaded to GCP). The gcloud suite also comes with a Flask server that you can run to execute and debug your code locally.

The architecture also shows the two external APIs that we will leverage: Coinbase Cloud and Trading View.

On the GCP side, we have the investment bot cloud function as the central element. On top, we have the cloud scheduler. The third key component is the Firestore document database.


Google Cloud Platform Suggested Architecture for crypto trading bot

In GCP, we use the following features:

  • Cloud Functions: This is the central component of our solution. Functions allow you to run Python code without deploying and maintaining any kind of infrastructure yourself. Google will take care of the whole management in the background. The great thing about Google (as opposed to AWS and Azure) is, that it allows you to deploy additional Python libraries together with your code. That is why I have personally chosen GCP over the other main cloud players. As of writing, Google uses Python 3.7 for their Cloud Functions.
  • IAM: We need Service Accounts (think of it as a technical user account) that can execute the code we deploy and the features we will use. Special permissions are then assigned to those Service Accounts. You do all of this in the IAM (Identity and Access Management) section of GCP. An example of a permission is the ability to execute Cloud Functions or to access a project’s Firestore.
  • Security: Since we do not want to expose sensitive Coinbase connection data as normal environment variables, we should create so called Secrets in the Secret Manager section. We will create two secrets, one for the Coinbase API key and the other one for the Coinbase API secret. Those secrets will then later be exposed as environment variables to our cloud function.
  • Cloud Scheduler: A Cloud Function will only execute when it is triggered externally, e.g. by receiving an HTTP request. The Cloud Scheduler allows us to define a regular time interval at which our Cloud Function is triggered. Therefore, we do not need to constantly trigger it by hand.
  • Firestore: This is a Google document database that is very easy to setup and use. We will use Firestore to keep track of buy and sell orders.

Trading Bot Framework

Here is the main Google Cloud function which calls two custom functions in line two and three. Both functions have a Pandas dataframe as an output. A Cloud Function cannot return a Pandas dataframe. Therefore, in line four, the dataframe is concatenated and outputted in JSON format and it is returned together with an HTTP 200 OK status:

def investment-bot(request):
df_buy_order_results = make_investment_decision()
df_sell_order_results = place_sell_orders()
return df_buy_order_results.to_json(orient='index')+df_sell_order_results.to_json(orient='index') , 200
view raw20230209_gcp_1.py hosted with ❤ by GitHub
Main Google Cloud Function of the crypto investment bot

The (simplified) sequence diagram of our crypto trading bot is as following:


Sequence diagram of crypto trading bot on GCP

The first function make_investment_decision() loads the necessary environment variables in lines 5 to 9 like the quote currency (e.g. USD or EUR) and base currencies (e.g. BTC, LTC, ETH) that you would like to use. In lines 10 to 14, the historic data, the 24 hour stats, server time and additional trading recommendations are requested via the Coinbase and Trading View APIs.

Next, the algorithm iterates through all crypto currencies you would like to trade (stored in environment variable MY_CRYPTO_CURRENCIES). Between line 17 and 26, the necessary data for decision making is prepared. The investment decision itself including a potential market order in Coinbase and the update to Firestore is done in lines 27 to 30.

def make_investment_decision():
import pandas as pd
import os
import json
IDLE_HOURS_BEFORE_NEXT_PURCHASE = float(os.environ.get('BOT_ONE_IDLE_HOURS_BEFORE_NEXT_PURCHASE'))
BOLLINGER_LOW_INVEST_EUR = float(os.environ.get('BOT_ONE_INVEST_EUR') or 0)
MY_CRYPTO_CURRENCIES = json.loads(os.environ['BOT_ONE_CRYPTO_CURRENCIES'])
MY_QUOTE_CURRENCY = os.environ.get('QUOTE_CURRENCY')
TRADING_VIEW_SYMBOLS = json.loads(os.environ['TRADING_VIEW_SYMBOLS'])
df_historic_data = cb_get_enhanced_history(MY_QUOTE_CURRENCY, MY_CRYPTO_CURRENCIES)
df_historic_data = df_historic_data.sort_values(['date'], ascending=True)
df_trading_view_signals = get_trading_view_signals(TRADING_VIEW_SYMBOLS)
df_24hstats = cb_get_24h_data(MY_QUOTE_CURRENCY, MY_CRYPTO_CURRENCIES)
server_time_now = cb_get_server_time()
order_results = []
for currency in MY_CRYPTO_CURRENCIES:
product_id = currency+'-'+MY_QUOTE_CURRENCY
current_value = df_24hstats.loc[currency]['last']
trading_view_recommendation = df_trading_view_signals.loc[currency]['1min_recommendation']
bb_low = df_historic_data.query('granularity == \'DAILY\' and base_currency == @currency').tail(1)['bb_low'].iloc[0]
last_buy_fill_date = cb_get_last_buy_fill_date(product_id)
idle_hours_reached = True
if last_buy_fill_date is not None:
last_fill_delta = ((server_time_now - last_buy_fill_date).days*86400 + (server_time_now - last_buy_fill_date).seconds)/3600
if (last_fill_delta < IDLE_HOURS_BEFORE_NEXT_PURCHASE):
idle_hours_reached = False
if (current_value < bb_low and idle_hours_reached and (trading_view_recommendation == 'BUY' or trading_view_recommendation == 'STRONG_BUY')):
order_id = cb_create_market_order(product_id=product_id, quote_size=BOLLINGER_LOW_INVEST_EUR)
fire_doc_id = fire_create_order_record(doc_id=order_id,doc_data={'buy_order_id': order_id, 'bb_low': bb_low, 'trading_view_recommendation': trading_view_recommendation, 'sell_order_id': ''})
order_results.append(f'Order now: {product_id}, price: {current_value}{MY_QUOTE_CURRENCY}, amount: {BOLLINGER_LOW_INVEST_EUR}{MY_QUOTE_CURRENCY} and order id {order_id} and Firestore doc id {fire_doc_id}')
else:
order_results.append(f'No order placed for {currency}; current value: {current_value}{MY_QUOTE_CURRENCY}; bb low: {bb_low}{MY_QUOTE_CURRENCY}; signal: {trading_view_recommendation}; last trade was on {last_buy_fill_date} ({idle_hours_reached})')
return pd.DataFrame(order_results)
view raw20230209_gcp_2.py hosted with ❤ by GitHub
The make_investment_decision() Python function

The next function place_sell_orders() initiates a sell order for each purchase order from the previous step. To achieve this, it iterates through all Firestore documents that do not yet have a sell order id (line 6 to 8). For each relevant document (one document is equivalent to one market buy order), missing order data is requested from Coinbase and Firestore is updated (line 9 to 33). Next, a sales order is created in Coinbase (line 34 to 41). At last, Firestore is again updated with the details of the sales order.

def place_sell_orders():
import pandas as pd
from datetime import datetime
import os
TARGET_MARGIN_PERCENTAGE = float(os.environ.get('BOT_ONE_TARGET_MARGIN_PERCENTAGE'))
docs = fire_get_orders_wo_sell_order_id()
order_results = []
for doc in docs:
# 1. Add missing data from filled buy orders (since data was not available when document was created)
buy_order_id = doc.to_dict()['buy_order_id']
filled_order= cb_get_aggregated_fills(order_id=buy_order_id).iloc[0]
buy_order_id = filled_order['order_id']
date = filled_order['date']
product_id = filled_order['product_id']
base_currency = filled_order['product_id'].split('-')[0]
quote_currency = filled_order['product_id'].split('-')[1]
buy_base_price = filled_order['price']
buy_base_size = filled_order['size'] / filled_order['price']
buy_quote_size = filled_order['size']
buy_quote_commission = filled_order['commission']
buy_quote_total_size = filled_order['total_price']
order_data = {'buy_date': date,
'buy_order_id': buy_order_id,
'product_id': product_id,
'base_currency': base_currency,
'quote_currency': quote_currency,
'buy_base_price': buy_base_price,
'buy_base_size': buy_base_size,
'buy_quote_size': buy_quote_size,
'buy_quote_commission': buy_quote_commission,
'buy_quote_total_size': buy_quote_total_size,
'sell_order_id': ''}
fire_doc_id = fire_create_order_record(doc_id = buy_order_id, doc_data = order_data)
# 2. Initiate Stop Limit sell order with respective targeet margin:
base_increment = cb_get_product_info(product_id)['base_increment']
base_decimals = get_decimal_places(base_increment)
quote_increment = cb_get_product_info(product_id)['quote_increment']
quote_decimals = get_decimal_places(quote_increment)
target_price = round(buy_base_price * (1 + TARGET_MARGIN_PERCENTAGE/100), quote_decimals)
buy_base_size = round(buy_base_size, base_decimals)
sell_order_id = cb_create_stop_limit_sell_order(product_id=product_id,base_size=buy_base_size,stop_price=target_price,limit_price=target_price)
# 3. Update Firestore document with sell order data:
fire_doc_id = fire_create_order_record(doc_id=buy_order_id,doc_data={'sell_order_id': sell_order_id,
'sell_base_target_price': target_price,
'sell_order_created_at': datetime.now(),
'target_margin': TARGET_MARGIN_PERCENTAGE/100})
order_results.append(f'Stop limit sell order {sell_order_id} for equivalent market buy order {buy_order_id} created with target price {target_price}{quote_currency} (target margin: {TARGET_MARGIN_PERCENTAGE}%); Firestore document updated ({fire_doc_id})')
return pd.DataFrame(order_results)
view raw20230209_gcp_3.py hosted with ❤ by GitHub
The place_sell_orders() Python function

In line 48, the list of sell orders is returned as a Pandas dataframe. Please take a look at the full code on my Github.

Prepare Google Cloud

Before we can test the investment-bot function locally and deploy it to the cloud, a few initial steps must be taken on GCP:

  • Create a Service Account to run Cloud Functions: To test the Firestore connection locally, you must create one Service Account that has the Firebase Admin SDK Administrator Service Agent permission.
  • Create a Service Account to access Firestore: To run the Cloud Function and Cloud Scheduler job, you must create one Service Account that has the Cloud Functions Invoker permission.
  • Initiate Firestore: In case you use Firestore for the first time in your project, Google will ask you to enable the feature. Make sure that you start Firestore in Native mode, otherwise the code examples from this article will not work.
  • Create two Secrets: Under the Secret Manager, you must create two secrets, one for the Coinbase API key and another one for the Coinbase API secret. Make sure that you assign the permission Secret Manager Secret Accessor to your Service Account from above.

Two Secrets created in the GCP Secret Manager

Test Code Locally

You can test the code locally via the built-in Flask server that comes with the functions-framework of the gcloud CLI. Run this command from the folder where the main.py file resides:

functions-framework --target=investment-bot --debug

When you run the command, it will show you the local IP address under which your cloud function is available. You just need to open it in a browser and you should see an output similar to this one:


Example response from investment function when executed locally

In case you run into errors, check the output of the terminal.

Deploy and Run Code on Google Cloud GCP

To deploy the investment bot to GCP, you must run the following command from the local folder containing the main.py file:

gcloud functions deploy investment-bot --runtime python37 --trigger-http --env-vars-file .env.yaml

If all goes well, you will see investment-bot as a new function within the Cloud Function section in GCP:


Deployed Python-based Cloud Function in GCP

Edit the function to add the Service Account that you have created above. Assign both secrets from the previous step. The secrets need to exactly match to the environment variables used within the Python code (ignore the Sendgrid secret since it is not used in the simplified trading bot example):


Three example Secrets exposed as environment variables within a GCP Cloud Function

You have the possibility to test the execution of the function from inside the function screen. If something goes wrong, check the log output. Typical errors that I have made in the past:

  • Forgot to assign a Service Account to the function which has the correct permission (Cloud Functions Invoker)
  • Forgot to associate the two Secrets properly, e.g. misspelling of the mapped environment variable names

Once your function executes successfully on GCP, you need to create a Cloud Scheduler job so that the function is executed on a regular basis. Go to the Cloud Scheduler section and create a new scheduler job. The only tricky part is to find out the exact unix-cron format for your purpose, here is the that executes every five minutes:

*/5 * * * *

Other than this, make sure that you again assign the same Service Account to the job and that you copy and paste the exact URL that represents your Cloud Function target. Select OICD as an authorization option. The job should look similar to this one:


Example Cloud Scheduler job with a five minute interval

That’s it! You can wait for five minutes and you should see the status of your scheduler job change to SuccessThis means that you can now relax and the investment bot will take care of your crypto trading all by itself.