Field Note

Serializing Protobuf Objects Using protoc

protobuf protoc serialization
Posted on Wednesday, May the 01, 2019
2 min read

Protobuf is awesome but what if we need to quickly debug or test a client?

This is especially helpful if you work with Twirp and code in Python since the code generator does not yield a JSON Protobuf client.

Solution

You can use the protoc command to generate binary data and then pass them to curl (see 1 and 2).

Switch to the location in the Gopath which contains your api.proto file:

cd $GOPATH/src/git.example.com/my-project/

Assuming a message type with fully qualified name example.Example

package example;

message Example {
  string data = 1;
}

we can generate a protobuf serialization by running

echo 'data: "bla"' \
| protoc --proto-path $GOPATH/src/git.example.com/my-project --encode example.Example api.proto

Piping this into curl call which expects binary data on STDIN lets us send protobuf serialized binary data to the target URL; e.g. for localhost:8000/twirp/example.MyRPCService/MyEndpoint

echo 'data: "bla"' \
| protoc --proto-path $GOPATH/src/git.example.com/my-project --encode example.Example api.proto
| curl -H "Content-Type: application/protobuf" --data-binary @- localhost:8000/twirp/example.MyRPCService/MyEndpoint

The input string for the encoder has a unique notation which is similar but not identical to JSON in that

Otherwise, the serialization rules are the same as given in the Protobuf language guide (e.g. messages map to objects, …).

Sources

friedrichkurz.me

© 2025 Friedrich Kurz

Privacy Policy