Skip to main content

Protocol Buffers!

Protocol Buffers are Google’s preferred method of representing and communicating structured data. For most Google projects, protocol buffers are used for data storage or client-server communication. As such, a working protocol buffer solution has been a requirement for J2ObjC from day one. Until recently our solution contained internal dependencies that prevented it’s public release, but now I am very pleased to be able to make our internal solution available to all J2ObjC users.

Let’s take a quick look at how protocol buffers are used (for a more in-depth look you can read through the Protocol Buffers Developer Guide). Suppose my app needs a geographic location, so I would create a geo.proto file with the following declaration:
message Location {
  optional string name = 1;
  optional double latitude = 2;
  optional double longitude = 3;

Then I can use the protocol buffer compiler, “protoc”, to generate data types in the languages I need:
$ protoc --java_out=src/java --cpp_out=src/cpp geo.proto
Now I have both a Java and a C++ class for my Location type that can be serialized to a language-independent binary form.

A working protocol buffer implementation for a particular language consists of two parts: the code generator, and an associated runtime. The code generator parses .proto files and generates code in the desired language. The runtime is a library implemented in the target language that provides serialization, base types, and any other support required by the generated types. Google protocol buffers support four languages: C++, Java, Python, and Ruby, but since protoc supports plugins, a plugin and runtime can be written for other languages and platforms.

There are actually several protocol buffer choices available for Java, all of which now work with J2ObjC:
  • Default protos - Provides the most feature-full environment with builder types for your messages and reflective features. A very rich solution, but perhaps a little bloated for a mobile application.
  • “Lite” protos - API compatible with the default protos, but requires only a small subset of the runtime library. Reflective features are stripped. See
  • javanano - An extremely lightweight implementation. Message types are generated without getter or setter methods, only public fields.
All of the above are supported by J2ObjC. For “javanano” protos the solution is simple: just translate the generated sources and runtime as you would any other Java sources. For the default (and lite) protos we provide a protocol buffer generator that creates Objective-C code from your .proto files that is compatible with generated Java code, and a Objective-C runtime library that is compatible with the Java runtime.

There are two reasons that we provide a custom J2ObjC implementation. The first is performance; in particular, fast serialization and deserialization. When an Inbox user opens the app, their entire inbox is stored as binary protocol buffer data and must be deserialized before the page can be rendered. Mobile app developers know that start-up time is critical. Unfortunately, the deserialization code in the Java protobuf library hits on one of J2ObjC’s main weaknesses: object creation. With a carefully tuned runtime, however, we’re able to outperform any translated code. One reason for this is that we were able to reuse parts of the C++ protocol buffer runtime that has been optimized over years of development.

The second benefit of the our custom J2ObjC implementation is code size. The generated Java protocol buffers can be quite bloated, especially when not using lite protos. We’re able to take advantage of Objective-C’s dynamic method resolution to avoid generating any field getter or setter implementations. All accessor methods are added dynamically by the message type’s base class. This helps minimize the footprint of generated data types.

Protocol buffers are an excellent alternative to XML or JSON. Consider using them to build your app’s data model and/or client-server interface. The binary serialization will save you space and the generated types make it easy to read, write, and edit your data. For more information about protocol buffers check out the developer site. For instructions on how to build protocol buffers with J2ObjC, see our wiki page.


Popular posts from this blog

Breaking Retain Cycles with @Weak and Other Techniques

The runtime on iOS and macOS uses reference counting to manage memory. When two objects make strong references to each other, a retain cycle is formed. Consider a Document object that has a Header. The Document owns the Header, but often the Header is also modeled to have a document (or “parent”) field:

cfront: a J2ObjC Inspiration

Developers new to J2ObjC may find its "Java compiles to Objective-C, which compiles to .o files" approach a little strange, but it's based on precedent: cfront. When C++ was first released, no compilers translated C++ sources directly into object files. Instead, the cfront script translated C++ into temporary C files, and then invoked cc to compile them. cfront took similar options as cc, so most C++ developers used it as if it was a true C++ compiler. The cfront script wrapped around a transpiler, though; it was a script very similar to combining the j2objc and j2objcc scripts in J2ObjC.
One of cfront's innovations was name mangling, where C++ type information was embedded into C names. J2ObjC also uses name mangling (described here) to support features like packages and Java's method overloading semantics that Objective-C doesn't directly support. For example, the JRE has both java.util.Date and java.sql.Date classes; since C names are all global in scope,…