Let’s Not Waste a Crisis!

The ongoing COVID-19 related suppression of economic activity will impact incomes across the board. Irrespective of how the income is generated (e.g. business, employment, self-employment) the impact can be either positive, negative or uncertain.

  • Positive for those whose incomes are not disrupted or are increased due to demand (e.g. PPE manufacturers, health-care staff, delivery drivers).
  • Negative for those whose incomes have been disrupted without any relief in sight (e.g. restaurants, people who have been laid off with bad prospects for getting another job).
  • Uncertain for those who have been furloughed or laid off but with good prospects for getting a job.

With anything between 6-11% contraction predicted, the majority of the cases should fall in the ‘Uncertain’ category (I predict 4-7%) who will move to either Positive or Negative category over the next year or so.

Why do I say that?

I say it because there will be different responses to the challenges, from restructuring, process improvements to failing fast and even retraining/reskilling (both at individual level and at an organisational level). Depending on how effective a business is at transforming itself to survive, a lot of the people in the ‘Uncertain’ category will quickly transition to the ‘Negative’ category.

One of the main transformation patterns is to carry out process improvements/restructuring with increased automation so that costs decrease and production/service elasticity increases as incomes fall initially but then recover over the medium and long term.

This group of people who jump from Uncertain to Negative is the BIG problem as this can trigger a long term contraction in consumption. How can we help these people reskill and retrain so that they can re-enter the job market? What can we do to support people as the pressure to automate increases as business income contracts?

Universal Basic Income

One possible answer to many of these questions is Universal Basic Income. If we provide people guaranteed support with basics (e.g. food, rent) then we are not only cutting them some slack but also decoupling ‘survival’ with ‘growth’.

Universal Basic Income (UBI) is a simple concept to understand: all citizens get a basic income every month irrespective of how much they earn. This is guaranteed from the day they turn 18 till the day they die. They may also get a smaller percentage from the day they are born to help their parents with their upkeep.

See this TED Talk by Rutger Bregman for more on this: https://www.youtube.com/watch?v=aIL_Y9g7Tg0

With UBI a recession will not impact the basics of any household. It will provide a safety net for families and individuals. It will also allow people to develop their skills and innovate.

There are a few wrinkles in this. Firstly, how should we prevent inflation as ‘free money’ is handed out to people? One proposed mechanism is to use a different class of money from the currency of the country. This UBI money cannot be used as a store of value (i.e. can’t be lent for interest), just for limited exchange (e.g. food, rent). This is similar to the US Supplemental Nutrition Assistance Program (SNAP) – also known as ‘food stamps’ (https://en.wikipedia.org/wiki/Supplemental_Nutrition_Assistance_Program)which can be exchanged for certain types of food. Many other countries have tried this experiment (such as Finland, USA, Canada etc.). This form of money should also ‘expire’ periodically so that people don’t start using them in a ‘money-like’ way.

Another challenge is how do you convert the ‘temporary’ UBI money into ‘permanent’ currency. This is required for the businesses accepting UBI money to be able to pass it down the supply chain (both locally and internationally). For example if you buy all your groceries with UBI money and it is not convertible to currency then how will the grocery shop pay it’s staff and suppliers. What if the suppliers were importing groceries from other countries – how would they convert UBI money to any international currency. In SNAP, the stamps are equivalent to money. It doesn’t have the same impact as UBI as its cost is a fraction of the total US GDP (0.5%).

Still, one should never let a good crisis go to waste! Time to think differently.

Currency Data, Efficient Markets and Influx DB

This post is about processing currency data which I have been collecting since the end of 2014. The data is collected once every hour from Monday 12am till Friday 11pm.

The data-set itself is not large as the frequency of collection is low, but it does cover lots of interesting world events such as Nigerian currency devaluation, Brexit, Trump Presidency, BJP Government in India, EU financial crisis, Demonetisation in India etc.

The image below shows the percentage change histogram for three common currencies (GBP – British Pound, USD – US Dollar and INR – Indian Rupee). The value for Percentage Change (X-Axis) is between -4% and 2%

Percentage Change histogram

Percentage Change histogram

What is immediately clear is the so called ‘fat-tail’ configuration. The data is highly skewed and shows clear features of ‘power law’ statistics. In other words the percentage change is related to frequency by an inverse power law. Larger changes (up or down) are rarer than small changes but not impossible (with respect to other distributions such as the Normal Distribution).

The discontinuity around Percentage Change = 0% is intentional. We do not want very small changes to be included as these would ‘drown out’ medium and large changes.

Mean Currency Movement

Mean Currency Movement

We can use the R code snippet below to draw 100 samples with replacement from  the movement data (combined across all currencies) and calculate the sample mean. The sample means can be plotted on a histogram which should give us the familiar Normal Distribution [this is the ‘Central Limit Theorem’ in action]. The sample mean that is most common is 0% – which is not an unexpected result given the presence of positive and negative  change percentages.

mean_curr_movement <- replicate(1000, { 
		sample(data$Percent.Change,100,replace = TRUE)

Compare this with a Normal distribution where, as we move away from the mean, the probability of occurrence reduces super-exponentially making large changes almost impossible (also a super-exponential quantity reduces a lot faster than a square or a cube).

Equilibrium Theory (or so called Efficient Market Hypothesis) would have us believe that the market can be modelled using a Bell Curve (Normal Distribution) where things might deviate from the ‘mean’ but rarely by a large amount and in the end it always converges back to the ‘equilibrium’ condition. Unfortunately with the reality of power-law we cannot sleep so soundly because a different definition of rare is applicable there.

Incidentally earthquakes follow a similar power law with respect to magnitude. This means that while powerful quakes are less frequent than milder ones they are still far from non-existent.

Another magical quality of such systems is that fluctuations and stability often come in clusters. The image below show the percentage movement over the full two years (approx.). We see a relative period of calm (green area) bracketed by periods of high volatility (red areas).

Movement Over Time

Movement Over Time

The above graph shows that there are no ‘equilibrium’ states within the price. The invisible hand has not magically appeared to calm things down and reduce any gaps between demand and supply to allow the price of the currency to re-adjust. Otherwise we would have found that larger the change larger the damping force to resist the change – there by making sudden large changes impossible.

For the curious:

All the raw currency data is collected in an Influx DB instance and then pulled out and processed using custom window functions I wrote in JAVA. The processed data is then dumped into a CSV (about 6000 rows) to be processed in R.

We will explore this data-set a bit more in future posts! This was to get you interested in the topic. There are large amounts of time series data sets available out there that you can start to analyse in the same way.

All the best!

Time-Series Dashboards with Grafana and Influx DB

I have been collecting currency data (once every hour) over the last couple of years, using a Node.JS retriever. The data goes in to a Mongo DB instance. We store the timestamp and all the currency pairings against the US Dollar (e.g. how may GBPs get you 1 USD).

Some other features we may need:

  • What if we want different pairings? Say we wanted to have pairings against Gold or Euros?
  • We may need to normalise the data if we want to compare the movement of different currencies?
  • We may also need ways of visualising the data (as time series etc.)

To cater to this I decided to bring in Grafana (I did not want to use the ELK stack because I believe if you don’t need text search don’t use elasticSEARCH!). Grafana does not support Mongo out of the box so I had to bring in Influx DB.

Bringing in Influx DB allowed me to integrate painlessly with Grafana (sort of). But as we all know – when you are integrating systems there is a certain minimum amount of pain you have to experience. Juggling around just shifts that pain to hopefully a location where you can deal with it efficiently. I my case I moved the pain to the integration between Mongo and Influx.

I had to create a component that pulls out the raw data from Mongo DB, pumps it through a pipeline to filter (to get the pairings of interest – the full data set has 168 currencies in it!), normalise and inject into Influx DB.

A side note: I used the Influx DB Java API which was REALLY easy to use, this encouraged me to go ahead with the Influx – Mongo integration in Java.

Grifana Influx

Grafana – Influx

I also wanted the Influx – Mongo integration to be ‘on-demand’ so that we can create different pairings against different targets – such as the big 3 against Gold, major currencies against SDR (International Monetary Fund – Special Drawing Rights) etc. and populate different time-series databases in Influx. So I gave the integration a REST interface using Spark-Java.

I did encounter one problem with Grafana – InfluxDB integration was the ‘easy’ query system did not work – I had to create one manually – which is pretty straight forward as the InfluxDB documentation is decent.


I was able to get the dashboards to work with normalised time-series of 6 currencies against Gold (XAU): US Dollar (USD), Indian Rupee (INR), Nigerian Naira (NGN), Euro (EUR), British Pound (GBP) and Chinese Yuan (CNY). I discovered something interesting when I was looking at the data around the ‘Brexit’ vote (23/24 June 2016).


The screenshot above is from Grafana. Filtered to show EUR-XAU, GBP-XAU, USD-XAU and NGN-XAU. We see a massive dip in Nigerian Naira before the actual Brexit vote. I was surprised so I googled and it seems that few days before Brexit vote the Naira was allowed to float against USD (earlier it was pegged to it) which led to a massive devaluation.

Then on Brexit day we see a general trend for the selected currencies to fall against Gold as it suddenly became the asset of choice once the ‘unthinkable’ had happened, with the GBP-XAU registering the largest fall. As you can see the Naira also registers a dip against Gold (XAU).

This also shows how universal Gold is. At one time all currencies were pegged against Gold, now unofficially USD is the currency of trade, but this shows when something unthinkable or truly ‘world-changing’ happens people run back to the safety of Gold.