Hello friend! This is a newsletter for builders.
What do you like to build?
I hope the ideas in this week’s newsletter can help you get it done.
In this weeks newsletter
⚔️ Challenge: Time estimations
💻 Tutorial: Python’s StringIO examples
⁉️ Stackoverflow: Embed GitHub code into an iframe
📜 Quote of the week
Challenge: Time estimations
Estimating project schedule timelines is difficult.
Next time you provide a time estimate, I challenge you to keep track of how close that turned out to be. If it was far off, then try to figure out where it went wrong.
It’s something I struggle with at times. So, next time I’m asked for an estimate, the first thing I’m going to say is “I’ll get back to you”.
Then I will:
Visualize the end result
Break the project down into pieces
Jot down an outline
Estimate time needed for each piece
Start building a prototype
When prototyping, I want to focus on the risky or more difficult parts early, so that I can get a better understanding of their time requirements.
Then I’ll provide the estimate ;)
If I’m really smart I’ll start keeping a log of these in Google Sheets to track their accuracy.
Tutorial: Python’s StringIO examples
I used StringIO to push data around at work the other day. It was a pipeline to move zip files on a remote server into a Postgres DB elsewhere.
So I decided to write up some basic examples to show off how cool and useful these object are.
So check it out.. Say I want to read a CSV file with Pandas. I could pass in the path:
df = pd.read_csv("fname.csv")
Or pass in the so-called “file-like” object
with open("fname.csv", "r") as f:
df = pd.read_csv(f)
Using StringIO I can create a file-like object from a string.
That lets me do things like this:
csv_obj = StringIO("""
food,score
apple,9
orange,2
kiwi,999
""")
df = pd.read_csv(csv_obj)
The actual use case I had for using StringIO was to stream a Pandas dataframe into Postgres:
csv_obj = StringIO()
df.fillna("").to_csv(
csv_obj,
index=False,
header=False,
sep="\t",
)
csv_obj.seek(0)
with psycopg2.connect(os.getenv("DB_URL")) as conn:
with conn.cursor() as c:
c.copy_from(csv_obj, "table_name", sep="\t", null="")
I create an empty file-like object, write the dataframe to it, and then seek the cursor back to the first byte of that data.
Then, in the 2nd chunk of code I connect to the database and copy it to the database. The function on the final line will call the COPY command and read from STDIN.
Here’s how this example looks on the Postgres database side:
Stackoverflow: Embed GitHub code into an iframe
Solution: https://stackoverflow.com/a/54101942/3511819
I doubt very many people have wanted to do this, but a couple years ago I did
It was tricky to figure out but an easy solution in the end.
The two notable points were to
Request the code from GitHub’s “api” subdomain and parse the result as JSON
fetch(
'https://api.github.com/repos/agalea91/crypto-monetary-base/contents/charts/relative_coin_supply_pct_estimates.html'
)
.then(
function(response) {
return response.json();
})
Set the HTML iframe source as base64 encoded content from the response
.then(
function(data) {
iframe = document.getElementById('github-iframe');
iframe.src = 'data:text/html;base64,' + data['content'];
}
);
If this caught your interest then check out the full working example.
Quote of the week
“You are not the person you were as a kid, 10 years ago, last year, or yesterday. You’ve learned from all that. You cannot dwell on the past. Quickly learn from past experiences and move on immediately. NEXT!”
- Jesse Stine
I read an investing book by Jesse Stine and learned his strategies for trading stocks. I was surprised how much general life advice was in there.
I read that while in Hawaii, dreaming of making it big on the stock market.
Since then I’ve been reading Jesse’s “Friends and family” newsletter and the quote above actually came from one of those posts.
It’s easy to get stuck dwelling on the past, but regrets are only useful so far as they can influence you to change future behaviours.
Either way, let yourself understand why you’re feeling that regret and extract any lesson there is. Then discard it as quickly as you can.
-Alex
Subscribe the Datapipe weekly newsletter ⬇️
Share this article ⬇️