How My Python Scripts Sound More Like a Natural Conversation
Your code is your documentation too.
People say that great programmers do not add comments to their code. They believe that if the code is hard to create, it should also be challenging for others to comprehend and alter. Hence they write straightforward code.
Although I don't advocate for not commenting at all, there's an element of truth in that statement I can't deny. The code should be readable by anyone!
This is why SQL codes are great. The declarative syntax is far more readable than any general-purpose Programming language. We know precisely what we select, the filters, how we aggregate, etc.
Could we organize our Python scripts for readability? If we could make our python code look more declarative, would that improve the code quality? And would it be more fun?
I've been experimenting with different code styles for some of my projects. And using pipe operators improves readability to an excellent level. Slowly, I'm converting most of my codebase to leverage this technique.
But before jumping into pipes,
What makes your codebase hard to read?
Evolution has made us create acceptable norms in every aspect of our everyday tasks. We might not know them consciously, but doing them differently makes life hard for us and those around us.
In programming, too, we've developed acceptable norms. Software design principles, patterns, and code style guides are such norms.
When we follow them in our code, readers can easily figure out the structure of our code. Failing to do so creates a hard time for you and the other person.
If these are new, check out SOLID design principles and the PEP 8 style guide.
One of the SOLID principles has particular importance in restructuring your code using pipe operators–The single responsibility principle (SRP). It says each block of our code(a function, a class, etc.) should handle one and only one aspect of our code.
Here's a code that violates the SRP:
import requests
def calculate_transaction_value(amount):
response = requests.get('https://api.exchangeratesapi.io/latest?base=USD&symbols=EUR')
exchange_rate = response.json()['rates']['EUR']
transaction_value = amount * exchange_rate
return transaction_valueThe above code queries an API for the latest exchange rate and calculates the transaction value in USD. Although this function is trivial, it handles two things – retrieving the exchange rate from the external API and computing the value.
There are several practical problems with this approach. Since this isn't the post's scope, I'll save it for another post. But most importantly, we can improve the code readability by introducing modularity.
Here's the same code after applying SRP (still not the other principles applied.)
import os
import requests
def get_exchange_rate():
url = "https://api.apilayer.com/exchangerates_data/latest?symbols=EUR&base=USD"
payload = {}
headers = {"apikey": os.environ["API_KEY"]}
response = requests.request("GET", url, headers=headers, data=payload)
result = response.json()["rates"]["EUR"]
return result
def calculate_transaction_value(exchange_rate, amount):
transaction_value = amount * exchange_rate
return transaction_valueHere's how you'd use it in your code:
exchange_rate = get_exchange_rate()
transaction_value = calculate_transaction_value(100, exchange_rate)
print(f'Transaction value in EUR: {transaction_value}')
>> Transaction value in EUR: 93.71199999999999Now, anyone can easily read the steps and understand them. The main code block is super simple. And whenever they need more detail, the can go into the function definitions.
Yet, when the code base grows large, which is the case in most real-life projects, keeping track of all the variables created and following along is difficult.
This is where pipes come in handy.
Pipes to boost the readability of our codebase.
If you work with shell commands, one of the amazing things is that we can pipe the output of one operation to the next. It gives a logical flow to the instructions.
We could also restructure our Python code to have that logical flow. We can do it with the pipe package. We can install it from PyPI:
pip install pipeWe can now convert our functions into pipe operators. We only have to annotate functions with the pipe decorator.
5 Python Decorators I Use in Almost All My Data Science Projects
import os
import requests
from pipe import Pipe
def get_exchange_rate():
url = "https://api.apilayer.com/exchangerates_data/latest?symbols=EUR&base=USD"
payload = {}
headers = {"apikey": os.environ["API_KEY"]}
response = requests.request("GET", url, headers=headers, data=payload)
result = response.json()["rates"]["EUR"]
return result
@Pipe
def calculate_transaction_value(exchange_rate, amount):
transaction_value = amount * exchange_rate
return transaction_valueHere's how we organize and call our functions in sequence.
transaction_value = (
get_exchange_rate()
| calculate_transaction_value(100)
)In the above example, note that the pipe operation automatically picks the function's first argument from its previous operation. We don't pass the exchange_rate argument to calculate_transaction_value function. Instead, we only pass the amount.
I intentionally kept this example straightforward. But in real-life projects, you'd have longer constructs.
Here's a modified extract from my recent project (still simplified.)
sales_regional_lead_data = (
get_sales_leads(region="EMEA")
| create_placeholder_dataframe(
years=3
) # Create a dataset replicating each sales lead for 12 months x years
| merge_budgets_to_sales_leads(
get_project_budgets() | aggregate_project_budgets_to_sales_leads()
)
| merge_crm_data_to_sales_leads(
get_crm_data() | aggregate_crm_data_to_sales_leads()
)
| merge_invoice_data_to_sales_leads(
get_project_invoices() | aggregate_invoices_to_sales_leads()
)
| merge_work_in_progress_to_sales_leads(
get_project_work_in_progress() | aggregate_work_in_progress_to_sales_leads()
)
| recognize_partial_invoices(finished_pct_cutoff=.8)
| compute_sales_for_each_sales_lead()
| compute_delivery_for_each_sales_lead()
| compute_margin_for_each_sales_lead()
| compute_average_margin_for_each_sales_lead()
| compute_average_delivery_for_each_sales_lead()
| compute_average_sales_for_each_sales_lead()
| load_sales_leads_to_database(
db_config, table_name="sales_leads", if_exists="append"
)
)This code version is easier to comprehend because it's closer to natural conversations. Anyone can understand the steps and parameters that alter the behavior of each step. You can show this even to your non-technical stakeholders, and they would happily read it as a novel.
The best way to organize our code is to break it into modules and keep them in separate files. Then these files are organized in a logical folder structure in the file system.
Therefore, only this code snippet lives inside my __init__.py file of the module. I import the functions from submodules. Thus, the folder structure looks something like this:
sales/
├── __init__.py
├── crm_data.py
├── invoices.py
├── budgets.py
├── work_in_progress.py
├── sales_leads.py
├── matrices.py
├── db.pyConclusion
As programmers, our first priority is often to get things working. But if that's the only goal, we're missing the point.
Our code should be easy enough to read and comprehend for others without our help or the need to go back and forth with our documentation pages constantly.
After trying several techniques, the one that struck me was pipe operations. This post is more about why I think this is great. But I haven't covered the full breadth of pipe operations here.
I've already written a comprehensive post about pipe operations. Please check it out for more ways to use it.
Use Pipe Operations in Python for More Readable and Faster Coding
Thanks for reading, friend! Say Hi to me on LinkedIn, Twitter, and Medium.
Not a Medium member yet? Please use this link to become a member because, at no extra cost for you, I earn a small commission for referring you.
