Fetch COVID-19 US state data and upload to site as JSON in Python 3.
Python 3 Script to grab COVID data and push it to a web-site
During the protests while already being locked down by the virus and curfews, maybe fear that the end was here (nah, we know better), I got bored and made a dashboard to watch the protest action from the safety of home as neither of us can risk the cost of COVID treatment or loss of licensing required for my wife’s career. In building this thing I decided to add the COVID-19 numbers for my state and this script was born!
This script will show basic use requests library to fetch data from a web-site, save that data as JSON, and upload the JSON to another web-site while displaying the values updated with JQuery.
#!/usr/bin/env python3
import requests # pip install requests
import datetime
import json
import time
import pysftp # pip install pysftp
'''
STATE COVID FETCHER
C. Nichols, June 2020
'''
now = datetime.datetime.now()
file_date_curr = now.strftime('%m-%d-%Y')
previous = now - datetime.timedelta(days=1)
file_date_prev = previous.strftime('%m-%d-%Y')
url_today = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports_us/%s.csv' % file_date_curr
url_yesterday = 'https://raw.githubusercontent.com/CSSEGISandData/COVID-19/master/csse_covid_19_data/csse_covid_19_daily_reports_us/%s.csv' % file_date_prev
urls = {
now: url_today,
previous: url_yesterday
}
state = 'ohio'
data = {'current':{}}
jdata = None
for dt,url in sorted(urls.items(),reverse=1):
req = requests.get(url)
if not req.status_code == 200: continue
print(dt,req.status_code)
for i in req.content.decode().split('\n'):
line = i.split(',')
if line[0].strip().lower().startswith(state):
confirmed = line[5]
deaths = line[6]
recovered = 0
if line[7].strip():
recovered = line[7]
active = line[8]
tested = line[11]
hospitalized = line[12]
data['current'] = {'date':dt.strftime('%m-%d-%Y'),'confirmed':confirmed,
'deaths':deaths,'recovered':recovered,'active':active,
'tested':tested,'hospitalized':hospitalized}
if data.get('current'):
jdata = json.dumps(data)
break
if jdata:
local_path = '/home/USERID/covid_ohio.json'
remote_path = 'json/covid_ohio.json'
host = 'SITE.COM' # You must have sftp setup for your domain.
usr = 'SFTP_USER'
pwd = 'SFTP_PASSWORD'
# Save JSON to local file system and upload to web-site if needed.
with open(local_path,'w') as fh:
fh.write(jdata)
time.sleep(1)
with pysftp.Connection(host, username=usr, password=pwd) as sftp:
sftp.put(local_path,remote_path)Once you have the JSON uploaded to your site you can use AJAX to populate HTML elements. I use JQuery so be sure to include that – it’s cleaner to look at, IMO. I set the update to run once a hour like so.
function getCovid() {
var covidJson = "json/covid_ohio.json";
$.ajax({
url: covidJson,
dataType: "json",
success: function (data) {
var deaths = data.current['deaths'];
var active = data.current['active'];
var hospitalized = data.current['hospitalized'];
var tested = data.current['tested'];
var confirmed = data.current['confirmed'];
$('#confirmed').html('<b>COVID-19 Confirmed:</b> <font color="blue">' + confirmed + '</font> <b>Active:</b> <font color="purple">' + active + '</font> <b>Hospitalized:</b> <font color="darkred">' + hospitalized + '</font> <b>Tested:</b> <font color="darkgreen">' + tested + '</font>');
},
error: function (result) {
alert("Error"+result);
},
cache: false
});
}
function tick() {
// Update once an hour...
var curr = new Date();
var mins = curr.getMinutes();
var secs = curr.getSeconds();
if (mins == 0 && secs == 0) {
getCovid();
}
}
$(document).ready(function() {
var timer = setInterval(tick, 1000);
getCovid(); // Load initial data.
});