Using google protobuf v2 in cmake projects on Windows

comments

Google Protobuf v3 library which have cmake support out of the box is in beta for now. What you can do if you'd like to use latest stable release of Google Protobuf on Windows? What if your project is cmake based? In this article I'll show the solution.

TL;DR

Standart way is wrong?

If you google around of using non-cmake projects within your cmake-based project, you'll find cmake's ExternalProject_Add was created right for this task. I've learned how to add external dependencies, but my opinion is this stuff is wrong on Windows.

First of all, lets clarify that cmake uses two-phase build process. In the first phase cmake will generate project files for build system specified (make on linux, Visual Studio projects on Windows by default). In the second phase you'll execute build system specified to actually build your project.

Google Protobuf v2 on Windows does not have installer, it provides only projects for Visual Studio 2005. It will be hard to create commands to configure (upgrade solution to Visual Studio version you'll use), build (run msbuild for solution), and install.

You could end up with this cmake command (this is not working sample):

ExternalProject_Add(protobuf
    PREFIX protobuf
    GIT_REPOSITORY "https://github.com/google/protobuf.git"
    GIT_TAG "v2.6.1"
    CONFIGURE_COMMAND "devenv ${CMAKE_CURRENT_BINARY_DIR}\\protobuf\\src\\vsprojects\\protobuf.sln /Upgrade"
    BUILD_COMMAND "msbuild ${CMAKE_CURRENT_BINARY_DIR}\\protobuf\\src\\vsprojects\\protobuf.sln"
    INSTALL_COMMAND " "
    TEST_COMMAND " "
)

But it will not works for you! So, what's wrong with ExternalProject_Add?

  • it actually download archives or clone remote repository too late - on second phase, when you actually build your project. The commands which is works in first phase (like FindProtobuf) will not find any files because they are not even downloaded!
  • it does not creates external library targets which can be referenced in your project.
  • it will install only single version of compiled binaries. This is not problem on linux, but is source of linker errors on Windows. There is multiple versions of CRT on Windows like dynamic or static, debug or release. And they are not binary compatible which is the root of problem.
  • it does not take into account cmake's generators/toolset/platform switches.

So, in general you cannot use ExternalProject_Add and then find_package. Weird.

You could build all configurations of protobuf separately, set PROTOBUF_SRC_ROOT_FOLDER variable and use find_package(protobuf REQUIRED). But it is bad practice that leads to the syndrome "works on my machine". What we really need is a simplified, completely reproducible build process.

Can we use the source?

The release is the point in time when the source code is stable and will not be changed. Google Protobuf has four releases tagged v2* in github repository: v2.4.1 (30 April 2011), v2.5.0 (27 February 2013), v2.6.0 (26 August 2014) and latest stable release is v2.6.1 (20 October 2014).

Why not to create custom CMakeLists.txt for that stable versions? They will not be changed with time, there is Visual Studio project files which contains all build information needed. It is easy to do, but a bit tedious.

The requirements for custom CMakeLists.txt is:

  • in generation phase do:
    • download sources from GitHub releases by tag
    • extract downloaded archive
    • run vsprojects/extract_includes.bat
    • copy the resulting include directory to public place
    • create targets for each protobuf library
  • in build phase:
    • build protobuf targets on par with your cmake-base project's targets
    • protobuf should be linked with your project with same version of CRT

All that stuff I created in cmake-external-packages project on GitHub.

cmake-external-packages basic usage

Use git-subtree, git-submodule or just copy repository contents to your repository subfolder.

To use protobuf, add following to your top-level CMakeLists.txt file:

set (EXTERNAL_PACKAGES_INCLUDE_DIR ${CMAKE_CURRENT_LIST_DIR}/third-party/include
    CACHE STRING "Directory for third-party include files, where include folders will be copied")
add_subdirectory(path/to/cmake-external-packages/protobuf-v2)
include_directories(${EXTERNAL_PACKAGES_INCLUDE_DIR})

#your add_subdiretory() or add_executable() here

By default, protobuf's includes will be copied to path/to/cmake-external-packages/include folder. In sample this location have been customized.

Just reference protobuf for your executable:

add_executable(your_exe ${your_exe_sources})
target_link_libraries(your_exe libprotobuf libprotobuf-lite libprotoc)

There is couple of configuration variables:

  • PROTOBUF_BUILD_SHARED_LIBS flag to decide to build protobuf libraries as static libraries or as shared dlls (OFF by default).
  • PROTOBUF_VERSION_TAG string to specify one of supported tags version (2.6.1 by default).

Known issues

Unfortunately, you cannot build Google Protobuf v2.4.1 and v2.5.0 with Visual Studio 2012 and higher. This is due to bug in STL which requires to #include <algorithm>. This was fixed in Google Protobuf v2.6.0 and higher.

Summary

If you need to consume Gooble Protobuf v2 on Windows, try cmake-external-packages.

And what the experience of using protobuf in Windows did you have?

Comments