It's been an year again, since I last posted, and this is going to be my third year in a row, where I've been promising regular updates, but failed to do so. I should accept, articulating my thoughts in a post is harder than working on an interesting project (which thankfully I've been working on a lot.)

2020 so far has been far from boring, each new day is worse than before and we still have half more to go. Hope everyone is staying safe and practicing Physical Distancing (I find Social Distancing to be a negative word)

Inspirartion

The project I am going to be talking about today has been on my backlog since 9 years, yes! nine!. This started back in summer of 2011 (my last year at university), Fuel Prices in Delhi were at an ATH, Twitter was the new trend in India, and I wanted to build a Twitter bot which could post daily price changes on twitter, which People could follow and see the change right in their feed. So what was stopping me from doing this then? well:

1. At the time, petrol prices were deregulated but diesel was still set by the government and rarely saw fluctuations.
2. There wasn't any reliable source of price information, let alone an API. Only news channels announced what the change was.
3. Servers were expensive, even though I was using HostGator shared servers at the time to host wordpress and other PHP projects, running a regular cron had it's limitations

Of these reasons, not having access to reliable information was the biggest bummer and this project was put on my backlog. I even had the account @FuelPriceHike registered for posting this in 2011. The fuel prices settled a bit and rallied again in 2014, but I was busy with other stuff then and did not bother with this. Come 2020 and this weird thing happened, fuel prices crashed across the world but India being the anomaly, the fuel prices kept on rising. It was an outrage everywhere, everyone was posting on how shitty the times were and what needs to happen, etc.

Wait a minute, I was reminded about this project I had planned almost a decade ago. Quick check on twitter and there it was, still waiting for me to finish this, quick password recovery and I have control of it again. I seemed to have gathered 68 followers randomly and At least a few hundered spammy events in my email to gain access to this account.

So, three weekends ago I sat down and explored what options are available for me to run such a bot in 2020 and believe me when I say things have drastically improved. I can now theoretically host such a bot for free, run it regularly and visualize this data in a spreadsheet with just a couple of lines of JavaScript. University me would've been amazed if I were to tell him I won't be writing PHP anymore and I also won't be using MySQL to structure the data, heck I won't even need to pay for hosting anymore.

Implementation

There are four essential parts to this project:

Fuel Prices API

A quick google search, sends me to economic times and they seems to host latest fuel prices. Interestingly I can just scrape these values? but wait, they actually have an internal api they hit for city and state prices. Sweet, this is so easy now!

Hosting Data

Google sheets at this point is my goto spreadsheet app, so I wanted this there too. Interestingly I've worked with Apps Scripts a few times to add custom stuff to my sheet. I was pretty sure, I could fetch an html document with the script and parse json. In 2020, Apps Script runs on the new V8 environment, which means I can do pretty much anything node can do or something pretty close. You can access the spreadsheet that hosts this data here. It is nothing fancy:

+-------------+-----------------------+-----------------------+
| Date        | Destination_Commodity | ...                   |
+-------------+-----------------------+-----------------------+
| Desc Date   | Price in INR          | ...                   |
+-------------+-----------------------+-----------------------+
| ...         | ...                   | ...                   |
+-------------+-----------------------+-----------------------+

Writing the bot

This is where interesting things are happening, I discovered Google's Clasp project It allows you build your appsscript project using your favourite IDE and transpile it to JS as need be. You can push and pull to your project like it were a git repo and now I can write custom build scripts to make it very simple to understand. The project can be found in this repo and this is very much a work in progress.

A quick `npm install` and `npm run build` will build the script in the `dist` folder ready for upload, `npm run push` pushes it to my Google Apps Script project (you can set that up easily by reading the getting started guide on the clasp project). The project is written using typescript and I use rollup to build one giant script which contains everything the script would need. The script is pretty straight forward and since I was going to use Google's custom types, static type-check helped me work faster.

The only hiccup I had was how twitter wants me to sign twitter module is decoupled and can be used as standalone by anyone who plans to post tweets using Google Apps Script.

Running the bot regularly

This is not a complete step by any means, but you can use `Triggers` in your apps script project, to trigger a function based on time (amongst other things). For this reason I expose a function called `runner` as the last step of my script, I can call this whenever I need to fetch prices and post on twitter. I set the trigger to run every 30 minutes.

Future Plans

This was a very simple proof-of-concept and it establishes Google Apps Script as a really powerful tool for such tasks. I am still working on this and plan to:

• Decouple API fetch mechanism, so in future I can plug in any other data source to fetch info.
• Complete the Unit Tests, custom Google Types and Methods need mocking and is PITA.
• Implement Github actions to push the build directly to my project on each commit.
• Implement insights like: Since when has this trend been happening, Sharing per destination graph, comparison of top N gainers/losers, etc.
• Interact with other similar tweets, responses, replies.

I know this has been a text heavy post, but I would highly recommend checkout out the repo: https://github.com/whizzzkid/fuelPrices

Share your own AppsScript comments in the comments below!

Stay Safe!