Build your protobuf stubs easier with a Makefile recipe

Build your protobuf stubs easier with a Makefile recipe

Welcome back to another politically incorrect baby monster blog. Today our niche use-case will see us familiarising ourselves with more cool tech lingo and real life examples.

Makefile

What's a makefile you say? A makefile is generally used to group and execute small snippets of code. A little bit like cmd aliases but they are project specific. For example, copy and paste the below in a file called Makefile at the root of your current python project

requirements:
	poetry export -o requirements-from-makefile.txt --without-hashes
    poetry export -o requirements-test-from-makefile.txt --without-hashes --dev
contents of Makefile

Now run...

make requirements

... and watch the two new requirement files appear in your current directory (assuming they weren't there already). Semantically, it's pretty nice too, although make dinner will probably not quite work in the same way, unless you have one of them robotic chef arms. And if you have one of them you are probably not a python developer and you are not reading this. Anyway... here's the gnu make manual if you are a total geek and have more than 5 minutes at your disposal.

Hang on, this failed, what's this poetry command?

Aaaah yes! Welcome to the newest and coolest packaging & dependency management tool for python. Check out poetry's basic usage if you have time. If not, carry on with a different example:

poetry:
	curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
contents of Makefile

Now execute:

make poetry

You now have poetry, which was not the purpose of this blog but pure cheekiness on my behalf because I like it so much. So SUE ME!

If you don't want to sue me, firstly, congratulations and secondly, please proceed to protobuf 101-101.

protobuf

This stands for protocol buffers and it's a google thing. When I first heard the term I thought of proto-buffoon, that can be also translated to the "earliest-form-of-clown" whatever that may mean. Despite the fact that it's a super cool concept, it always sounded weird inside my head.

Now back to business. Imagine you have to write an api to serve a model for an e-commerce website. You could start by either (a) writing the code first or by (b) writing the schema definition first or (c) something other than (a) or (b) or (d) ... ok stop.

The code-first approach will get you up and running super quickly and it will slow you down later when you have to make changes or write documentation. The schema-definition first approach "may" slow you down in the beginning and will get you self-documentation and api discoverability from the get go. Not to mention that it's so much easier to hand over and knowledge share. The benefit of using protobuf as part of the schema-definition approach is that... drumroll...

I don't care if this is not a drumroll because it sounds better than a f*cking drumroll

... the protobuf compiler "creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format."

So this proto-buffoon thing writes code for you - yes. Anyway, this is a story for another time. In the meantime here's a very simple example. Create a directory called protobuf and in there a file called restaurant.proto with the below contents:

syntax = "proto3";

package = myapi.v1;

message Restaurant {
	int32 restaurant_id = 1;
    string restaurant_name = 2;
}
contents of protobuf/restaurant.proto

Then go back to your Makefile and add the below:

PROTO_DIR=proto
STUBS_DIR=stubs

stubs:
	rm -rf $(STUBS_DIR) 2>/dev/null
    mkdir $(STUBS_DIR)
    python -m grpc_tools.protoc \
    -I $(PROTO_DIR) \
    --python_out=$(STUBS_DIR) \
    --grpc_python_out=$(STUBS_DIR) \
    $(shell find $(PROTO_DIR) -iname "*.proto")
    

Instead of me unpacking the above, ask yourself the below 5 questions and see if you can work the answers out:

  1. How can I define and access a variable in a Makefile?
  2. How can I suppress error messages in bash?
  3. How can I find a file in a directory?
  4. How can I write a multiline command?
  5. How can I execute a bash command inside a Makefile?

Finally, get to your terminal and type:

make stubs

Check the stubs directory for your goodies. That's it!

TLDR

In short, use a Makefile to give your hands a rest by reducing your repetitive key-strokes, in our use case, generating stubs from protobuf. This reduced 212 keystrokes to 10. Take it from someone with RSI, every little helps. However, your hands are not the only parts of you that need a rest. Your mind needs downtime too. Check this très cool podcast by python and mindset coaches Bob Belderbos and Julian. Sequeira.

#035 - Time for a Break! - PyBites Podcast
We’re doing it differently this week! Even though Bob was on holidays last week, he jumped on to have a chat... so naturally Julian quizzes him on the benefits of taking a break!The underlying message / takeaway: Take a long break and make sure yo...

Until next time.

あなたのフレンドリーな近所のプログラマー