Datapipe Weekly #26
In this weeks newsletter
🧘 Mind: 5 liters / minute
💻 Code: Pandas explode on URL parameters
❤️ Gem: Generative art
📜 Quote: Shell gas station
🧘 5 liters / minute
Breathe in… think “I am breathing in” - and breathe out… think “now I am breathing out.”
Your heart pumps ~5 liters of blood a minute. If you still your body and your mind then you can feel it.
Bring your entire focus on the heartbeat, then extend it to the surrounding area and on down your arms. Can you feel it flowing around - keeping you alive?
💻 Pandas explode on URL parameters
Last week I was doing a parameter extraction for a bunch of URLs.
If you don’t think much about webpage URLs then you may have never noticed these parameters before. Here’s an example:
https://www.adidas.ca/en/search?division=shoes&producttype_en_ca=athletic_sneakers
This URL has two parameters: division
and producttype_en_ca
.
So here’s how I extracted these for a big list of URLs:
Load the data
>>> python
Python 3.7.4 (default, Aug 13 2019, 15:17:50)
import pandas as pd
df = pd.DataFrame({
"url": [
'https://www.adidas.ca/en/search?division=shoes&producttype_en_ca=athletic_sneakers',
'https://www.adidas.ca/en/search?division=shoes&producttype_en_ca=casual'
]
})
Extract params
df["query_params"] = df.url.astype(str).apply(lambda x: x.split("?")[-1].split("&") if "?" in x else [])
“explode” the dataframe
df_boom = df.explode("query_params")
Notice in the final screenshot how we now have 4 rows in the “exploded” DataFrame compared to just 2 before. The index and other columns (in this case just ‘url’) get duplicated to pad out the extra rows.
Pretty neat trick eh? Now all you need is a little df.query_params.apply(lambda x: x.split(“=”)[0]).value_counts().sort_values(ascending=False).to_csv(“top_query_param_keys.csv“)
and bob’s your uncle!
❤️ Generative art
Speaking of ‘explosions’ - generative art on Ethereum is going wild right now
See for yourself:
📚 NFTs bring to mind ‘the metaverse,’ a major topic of the book I’m reading right now - Snow Crash by Neal Stephenson
📜 Quote
'When I look at people now, I don't see issues,' he said. 'I see souls.' The man said he had a spiritual experience. Actually, he said he had four.
He didn't go to the mountains, or the ocean, or the desert to have them. He had his four spiritual experiences in the same place - in the parking lot outside a Shell gas station in Portland, Oregon.
- Melody Beattie (Journey to the Heart)
I don’t believe in anything resembling a soul, instead I perceive human life to be no more special than any other - dog, monkey, cow - except for a few extra brain cells.
That said, I do believe in spiritual experiences, in so far as my perspective can allow. I’ve never had one of these at the gas station, but then again - I’ve never had one on the mountain either.
Spirituality aside,
What do you see when you look at people? Do you see their flaws? Their beauty? What they owe you? What you owe them?
What if you could see none of this - see right through. See instead the light within.
-Alex
Thank you for reading Datapipe 👋
This is a newsletter for builders!
What do you like to build?
I hope the ideas in this week’s newsletter can help you get it done.
Subscribe the Datapipe weekly newsletter ⬇️