"vscode:/vscode.git/clone" did not exist on "bdd17998e60f6151bf6d1f7a8e1cadfb73a0aaa4"
Commit 4cdfc960 authored by Vadim Markovtsev's avatar Vadim Markovtsev Committed by Vadim Markovtsev
Browse files

Simplify and improve fastprep.mk

There is no need in building Tensorflow from scratch, fastprep successfully links with the Python wrapper from pip and system protobuf v3. Tested this on Ubuntu.
parent bc70271a
...@@ -16,50 +16,16 @@ ...@@ -16,50 +16,16 @@
# limitations under the License. # limitations under the License.
# This makefile builds "fastprep", a faster version of prep.py that can be used # This makefile builds "fastprep", a faster version of prep.py that can be used
# to build training data for Swivel. Building "fastprep" is a bit more # to build training data for Swivel.
# involved: you'll need to pull and build the Tensorflow source, and then build
# and install compatible protobuf software. We've tested this with Tensorflow
# version 0.7.
#
# = Step 1. Pull and Build Tensorflow. =
#
# These instructions are somewhat abridged; for pre-requisites and the most
# up-to-date instructions, refer to:
#
# <https://www.tensorflow.org/versions/r0.7/get_started/os_setup.html#installing-from-sources>
#
# To build the Tensorflow components required for "fastpret", you'll need to
# install Bazel, Numpy, Swig, and Python development headers as described in at
# the above URL. Run the "configure" script as appropriate for your
# environment and then build the "build_pip_package" target:
#
# bazel build -c opt [--config=cuda] //tensorflow/tools/pip_package:build_pip_package
#
# This will generate the Tensorflow headers and libraries necessary for
# "fastprep".
#
# #
# = Step 2. Build and Install Compatible Protobuf Libraries = # = Step 1. Install protobuf v3 =
# #
# "fastprep" also needs compatible protocol buffer libraries, which you can # Ubuntu 16.10+: sudo apt install libprotobuf-dev
# build from the protobuf implementation included with the Tensorflow # Ubuntu 16.04: https://launchpad.net/~maarten-fonville/+archive/ubuntu/ppa + replace xenial with yakkety in /etc/apt/sources.list.d/maarten-fonville-ubuntu-ppa-xenial.list
# distribution: # macOS: brew install protobuf
# #
# cd ${TENSORFLOW_SRCDIR}/google/protobuf # = Step 2. Build "fastprep". =
# ./autogen.sh
# ./configure --prefix=${HOME} # ...or whatever
# make
# make install # ...or maybe "sudo make install"
#
# This will install the headers and libraries appropriately.
#
#
# = Step 3. Build "fastprep". =
#
# Finally modify this file (if necessary) to update PB_DIR and TF_DIR to refer
# to appropriate locations, and:
# #
# make -f fastprep.mk # make -f fastprep.mk
# #
...@@ -68,20 +34,27 @@ ...@@ -68,20 +34,27 @@
# matrices and other files necessary to train a Swivel matrix. # matrices and other files necessary to train a Swivel matrix.
# The root directory where the Google Protobuf software is installed. CXXFLAGS=-std=c++11 -march=native -g -O2 -flto -Wall -I.
# Alternative locations might be "/usr" or "/usr/local". LDLIBS=-lprotobuf -pthread -lm
PB_DIR=$(HOME)
FETCHER=curl -L -o
TF_URL=https://github.com/tensorflow/tensorflow/raw/master
PROTOC=protoc
%.proto: tensorflow/core/example
$(FETCHER) $@ $(TF_URL)/$@
%.pb.cc: %.proto
$(PROTOC) --cpp_out=. $<
# Assuming you've got the Tensorflow source unpacked and built in ${HOME}/src: fastprep: fastprep.cc tensorflow/core/example/feature.pb.cc tensorflow/core/example/example.pb.cc
TF_DIR=$(HOME)/src/tensorflow
PB_INCLUDE=$(PB_DIR)/include tensorflow/core/example:
TF_INCLUDE=$(TF_DIR)/bazel-genfiles @mkdir -p tensorflow/core/example
CXXFLAGS=-std=c++11 -m64 -mavx -g -Ofast -Wall -I$(TF_INCLUDE) -I$(PB_INCLUDE)
PB_LIBDIR=$(PB_DIR)/lib clean:
TF_LIBDIR=$(TF_DIR)/bazel-bin/tensorflow/core @rm -f fastprep
LDFLAGS=-L$(TF_LIBDIR) -L$(PB_LIBDIR)
LDLIBS=-lprotos_all_cc -lprotobuf -lpthread -lm
fastprep: fastprep.cc mrproper: clean
@rm -rf tensorflow
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment