Commit d7281311 authored by Derek Mauro's avatar Derek Mauro
Browse files

Merge pull request #3189 from ellert:gtest-help-test-GNU/kFreeBSD

PiperOrigin-RevId: 349349288
parents 389cb68b ed1bf868
# How to Contribute
googletest development is Piper-First. Just create a regular Piper CL. When the
CL is accepted and submitted, it will make its way to OSS via regular releasing
process.
# gMock - a Framework for Writing and Using C++ Mock Classes
<!--#include file="under-construction-banner.md"-->
**Status:** Draft \
**Tiny URL:** http://go/gmockdesign \
**Author:** Zhanyong Wan (who/wan)
<!-- GOOGLETEST_CM0035 DO NOT DELETE -->
(To participate in discussions on gMock, please subscribe to
[opensource-gmock](https://groups.google.com/a/google.com/group/opensource-gmock/subscribe).
Past discussions can be viewed
[here](https://groups.google.com/a/google.com/group/opensource-gmock/topics) and
[here](https://mailman.corp.google.com/pipermail/c-mock-dev/).)
(The slides for my gMock preview talk can be found here:
[ppt](http://wiki.corp.google.com/twiki/pub/Main/WanTalks/0706-beijing-gmock-preview.ppt).)
## Objective
[Mock objects](http://en.wikipedia.org/wiki/Mock_object) are simulated objects
that mimic real objects in controlled ways. They are useful for driving the
design of a system, and for testing some other object when it's difficult to use
real objects in a test.
While the idea of mocks applies to all objected-oriented languages, writing them
in C++ has many practical difficulties, due to the lack of support for
reflection in the language, the complexity and irregularity of C++, and the lack
of adequate tools. As an unfortunate result, C++ programmers often avoid writing
mocks, resulting in big, monolithic classes in production, and slow, brittle,
and difficult-to-maintain tests.
We believe that a good framework can make it much more pleasant to write and use
mocks in C++. Such a tool would help people write more
[small](https://wiki.corp.google.com/twiki/bin/view/Main/GoogleTestDefinitions)
tests that are quick, robust, and precise. Perhaps more importantly,
incorporating mocks early and often in the design process helps people discover
the role interfaces in the system and thus often leads to
[better designs](http://www.jmock.org/oopsla2004.pdf).
We plan to develop *gMock* as a generic framework for creating and using mock
classes in C++. We would encourage people to *use gMock as a design tool as much
as a testing tool*.
### Goals of gMock
* **Supporting all interfaces:** A user should be able to use gMock to create
a mock class for any C++ interface (i.e. a class whose methods are virtual).
In particular, interface templates should be supported, and there should be
no restriction on the types of the parameters - `const` parameters, pointer
parameters, reference parameters, reference-to-`const` parameters, and etc
should all be allowed.
gMock can also be used to mock a "loose" interface (i.e. the set of
operations a template class or template function expects its type argument
to support). This is useful for testing code that uses the
["high-performance dependency injection"](https://engdoc.corp.google.com/eng/doc/tott/episodes/33.md)
technique.
* **Precise specification of the intention:** gMock should enable a user to
precisely specify the intended behavior of a mock object, and how its
methods are expected to be called (in what order, with what arguments, etc).
In particular, it should not force the user to over-specify the problem
(which results in brittle tests that break when unrelated changes to the
code are made), or to under-specify the problem (which results in tests that
continue to pass when they shouldn't).
* **Intuitive and declarative syntax:** A declarative syntax fosters thinking
at the right abstraction level, and makes the code readable and less
error-prone. Therefore gMock should provide intuitive and declarative syntax
for:
1. creating a mock class, and
2. controlling the behavior of a mock object. When the two goals conflict,
the latter takes precedence, as it usually needs to be done many more
times than the former.
* **Extensible:** No framework can be expected to cover all users' needs.
Therefore, gMock shouldn't tie the users to whatever it provides. Instead, a
user should be able to easily extend the framework to accomplish more
advanced tasks.
* **Helpful error messages:** Bad error messages are a sure-fire way to
frustrate the users and drive them away. Therefore, gMock should generate
clear and sensible messages
1. when the code fails to compile - this can be hard as lots of templates
have to be used in the implementation, but we should try our best; and
2. when a user-supplied expectation fails. This also applies to
user-defined extensions, given that the user has done a good job
implementing the extensions.
* **Easy to learn:** We want gMock to make people's life easier, not harder.
It defeats our purpose if the framework is complex and difficult to learn.
* **Easily automatable:** The design of gMock should make the process of
creating a mock class from an interface fairly mechanical, and thus doable
by the automated
[mock class generator](https://wiki.corp.google.com/twiki/bin/view/Main/MockClassGeneratorDev).
* **Working in Google's environment:** While we may be interested in open
sourcing gMock later, our primary goal is to serve Google. Therefore gMock
must work well in our environment. In particular, it must not use
exceptions, and should work well with
[gUnit](https://wiki.corp.google.com/twiki/bin/view/Main/GUnitGuide).
### Non-goals
* **Mocking non-virtual methods:** gMock is a source-level tool that works
with standard compilers and linkers. It doesn't attempt to swap the object
code of a mock class and that of a real class on-the-fly. Therefore, only
virtual methods and template arguments can be mocked by gMock.
* **Supporting arbitrary number of parameters:** Due to limitations of the C++
language, there will be a practical limit on the number of parameters a mock
function can have. Support for more parameters can be added as needed.
* **Supporting non-Linux platforms:** The initial implementation may not run
on Windows or Mac OS. We have limited resources and need to make sure that
Linux users are served first. However, we'll try to avoid doing things that
will make porting gMock to non-Linux platforms difficult.
* **Special support for particular projects:** gMock is a generic framework
that makes mocking easy for *all* Google C++ projects. It should not contain
logic that's useful only to a small number of projects.
## Background
### Terminology
Different people often use "mock" to mean different things. This document
borrows the terminology popularized by
<a href="http://www.martinfowler.com/articles/mocksArentStubs.html">Martin
Fowler</a>:
* **Dummy** objects are passed around but never actually used. Usually they
are just used to fill parameter lists.
* **Fake** objects actually have working implementations, but usually take
some shortcut (perhaps to make the operations less expensive), which makes
them not suitable for production.
* **Stubs** provide canned answers to calls made during the test, usually not
responding at all to anything outside what's programmed in for the test.
* **Mocks** are objects pre-programmed with expectations which form a
specification of the calls they are expected to receive.
### Fakes vs Mocks
Many people are not clear about the difference between a fake and a mock, and
use the terms interchangeably. However, to understand why we need gMock and what
it will deliver, it's crucial to distinguish the two.
Compared with a fake, a mock object is "dumb" and usually doesn't have a working
implementation. However, it allows the user to control its behavior and set
expectations on the calls it will receive. For example, you can tell a mock
object that its `Foo()` method will be called twice with an argument that's
greater than 10, and it should return 12 and 14 respectively.
It may seem that mocks are not very useful compared to fakes, but the Java
community has shown this perception to be wrong. The ability to control a mock
object's behavior and specify the expected interaction between it and the code
under test makes it far more flexible and useful than a fake in designing and
testing a software system.
While fake classes have to be crafted with domain knowledge, mock classes can
actually be created mechanically - with suitable support from a framework. In
more dynamic languages like Java, C#, and Python, there are tools that create
mock objects on the fly without any user intervention. In C++, this cannot be
done within the language itself. However, a framework can make the task much
easier for a user, and the
[mock class generator](https://wiki.corp.google.com/twiki/bin/view/Main/MockClassGeneratorDev)
will make the process automated to a large extent.
### C++ Mocking at Google
To our knowledge, no tool or library is used at Google to facilitate the
creation of mock classes. As a result, people have been writing mock classes
manually. Such classes are typically tedious to create, and lack functionalities
for effective mocking. As a result, people are often frustrated and decided to
avoid mock classes.
As a rough estimate, as of 3/15/2007, the number of existing C++ mock classes in
our source tree is:
```shell
$ gsearch -f="\.(h|cc|cpp)$" -a -c "^\s*class\s+(Mock\w*|\w+Mock)\s*[:{]"
748
```
while the number of all C++ classes is:
```shell
$ gsearch -f="\.(h|cc|cpp)$" -a -c "^\s*class\s+\w+\s*[:{]"
188488
```
Roughly 1 out of every 250 C++ classes has a corresponding mock class. Clearly
this is not enough.
### Situation outside Google
The situation of using C++ mocks outside of Google is not a lot brighter either.
Although there is an open-source framework
([mockpp](http://mockpp.sourceforge.net/)) for writing mock classes, it is
overly complex and has limited functionalities. As a result, it doesn't have a
large following.
### Existing Mock Frameworks
A good mock framework takes years of hard work and actual use in the field to
mature. Therefore, it pays hugely to learn from existing mock frameworks: what
they can and cannot do, why they are the way they are, how they have evolved,
what lessons their creators have learned, and what they intend to do next, etc.
We studied some well-known mock frameworks for Java
([Mock Objects](http://www.mockobjects.com),
[EasyMock](http://www.easymock.org), [jMock 1](http://www.jmock.org), and
[jMock 2](http://cvs.jmock.codehaus.org/browse/~raw,r=1.3/jmock/website/content/cheat-sheet.html))
and for C++ ([mockpp](http://mockpp.sourceforge.net/)). Our conclusion is:
* Mock Objects is the most primitive of the four. It provides some basic
constructs for a user to set expected arguments and return values, but not
much beyond that.
* EasyMock makes the simple case very easy, but isn't flexible enough to
handle more advanced usage well. Often the users are forced to either
over-specify or under-specify their intention, resulting in brittle or
imprecise tests.
* jMock 1 and 2 share the same design philosophy, but have incompatible
syntaxes. They allow a user to precisely specify the intention of the test
in most cases, and can be easily extended by the user to handle more complex
situations.
* mockpp is a mixed bag of constructs from the above three. It doesn't have a
coherent design philosophy, and doesn't address C++'s specific requirements
well. It is more complex, redundant, and difficult to learn and use than we
would like.
### Our Plan
We believe that jMock is the most interesting and promising of the four. Its
creators have been aggressively experimenting with new ideas and designs, and
have produced many iterations before the current form. They have also documented
their experience and lessons in developing jMock in
[two](http://www.jmock.org/oopsla2004.pdf)
[papers](http://mockobjects.com/files/evolving_an_edsl.ooplsa2006.pdf), which
contain many valuable insights.
Therefore, the design of gMock is heavily influenced by jMock. Many constructs
will be directly ported from jMock. Meanwhile, we'll revisit various design
decisions in C++'s context to make sure that we take advantages of C++ strengths
and avoid its weaknesses. We will also address some challenges that are unique
to C++.
## Overview
### Why a Framework
Mock objects serve two distinct purposes in designing and testing a software
system:
1. They implement the same interfaces as the real classes and provide canned
responses, allowing code that uses them to compile and run; and
2. They can verify that the actual interaction between them and the code under
test matches what the user expects (for example, the right functions are
called, in the right order, with the right arguments, etc).
Without a framework, a user could manually implement mock functions to return
answers that are either pre-defined or computed using simplified logic. To
verify that the interaction that actually happens matches the expectation, a
user would typically let the mock functions record the interaction in some way,
and inspect the record in the end. This poor man's approach leaves several
things to be desired:
1. Writing a mock class manually is not easy, and often viewed as a burden to
be avoided.
2. Different tests use a mock class in different ways. Therefore, it is often
impractical to provide a working fake implementation that is useful for all
tests.
3. Describing what the interaction should be by inspecting what really has
happened is round-about and unnatural. It obscure the intention of the test
author, and results in tests that are hard to read and understand.
4. It is often too late to check how the interaction went after it has
finished. Much better is to report a failure at the exact moment an
expectation is violated. This gives the user a chance to check the context
of the failure (the stack trace, the variables, etc) before important
information is lost.
The purpose of gMock is to address the above problems. In particular, it will:
1. make the task of writing a mock class much easier by hiding the low-level
mechanism from the user;
1. let the user of a mock class, rather than its creator, specify the intended
responses;
1. let the user specify the intended interaction in a clear and direct syntax;
and
1. catch violations to the specification as soon as they arise.
### gMock's Expressiveness
The Java community's years of experience using mocks shows that a mock framework
should enable a user to directly specify the following properties of the
interaction between a mock object and its surrounding code:
* How many times will a function be called?
* What arguments will be used?
* In what order will the calls be made?
* What should the functions return?
* What side effects (if any) should the calls incur?
Also, it's important to be able to loosen the constraints when necessary to
prevent brittle tests. For example,
* If the test doesn't care about how many times a function will be called, the
test writer should be able to make that clear;
* If the exact value of an argument doesn't matter, the user should be able to
say so;
* If only a subset of the calls need to happen in a strict order, the user
should be allowed to specify a partial order.
### Architecture of gMock
gMock is a C++ library that will be linked into a user's test code. It consists
of the following components (the syntax used in the code samples is
*tentative*):
1. **Function mockers:** A family of template classes will be provided for the
user to mock functions with different arities. For example, a field of type
```
FunctionMocker<int(bool, const string&)>
```
will be used to mock a function with signature
```
virtual int Foo(bool, const string&);
```
1. **Specification builder:** This provides a syntax for the user to specify
the expected arguments and responses of a mock function. For example, to say
that `Foo()` will be called exactly twice with arguments `true` and a string
that contains `"hello"`, and will return 10 and 12 respectively, the user
can write:
```
EXPECT_CALL(mock_object, Foo(Eq(true), HasSubstring("hello"))
.Times(2)
.WillOnce(Return(10))
.WillOnce(Return(12))
```
1. **Cardinalities, matchers, and actions:** A collection of pre-defined
*cardinalities* (e.g. `2`), argument *matchers* (e.g. `Eq()` and
`HasSubstring()`), and stub *actions* (e.g. `Return()`) will enable the user
to precisely specify the intended interaction in most cases. When this set
is inadequate, the user can easily define new cardinalities, matchers, and
actions.
1. **Specification interpreter:** An underlying interpreter will verify that
the actual calls made to the mock object conform to the user's expectations.
gMock helps a user in two kinds of activities: *writing* mock classes and
*using* them in tests. When writing a mock class, a user employs the function
mockers (#1); when using a mock class, the user relies on #2 and #3 to specify
the expected interaction between the mock object and the code under test. As the
test runs and the mock functions are invoked, the specification interpreter (#4)
verifies that the actual interaction matches the expectation, and fails the test
when the two don't match.
## Detailed Design
### Implementing a Mock
This section explains how a user would implement a mock class using gMock. The
final syntax may be slightly different to what's presented here, but the overall
idea should remain the same.
The goal of the design is to allow mocking functions that take 0 or more
arguments, functions that are overloaded on the number/types of parameters,
const methods, and methods that are overloaded on the const-ness of this object.
#### Using macros
The easiest way to define a mock class is to use the `MOCK_METHOD` macro.
Specifically, to mock an interface
```cpp
class FooInterface {
...
virtual R Method(A1 a1, A2 a2, ..., Am am) = 0;
virtual S ConstMethod(B1 b1, B2 b2, ..., Bn bn) = 0;
};
```
one would simply write
```cpp
class MockFoo : public FooInterface {
...
MOCK_METHOD(R, Method, (A1 a1, A2 a2, ..., Am am), (override));
MOCK_METHOD(S, ConstMethod, (B1 b1, B2 b2, ..., Bn bn), (const, override));
};
```
#### Using no macro
The user can also choose to implement a mock class without using the macros.
For each function to be mocked that is not overloaded, the user should define a
**function mocker** member variable and implement the function by forwarding the
call to the function mocker, which knows how to respond to the given arguments.
A user specifies the mock function's default behavior and expectations on it by
calling the *mock spec function* in an `ON_CALL()` or `EXPECT_CALL()` statement.
Now let's see the concrete syntax. To mock a function that takes no argument:
```cpp
class AbcInterface {
...
virtual R Foo() = 0;
};
```
a user would write:
```cpp
class MockAbc : public AbcInterface {
...
// Mock Foo(). Implements AbcInterface::Foo().
virtual R Foo() { return gmock_Foo.Invoke(); }
FunctionMocker<R()> gmock_Foo;
};
```
To mock a function that takes some arguments:
```cpp
virtual R Bar(A1 a1, A2 a2);
```
a user would write:
```cpp
virtual R Bar(A1 a1, A2 a2) { return gmock_Bar.Invoke(a1, a2); }
FunctionMocker<R(A1, A2)> gmock_Bar;
```
To mock a `const` method:
```cpp
virtual R Baz(A1 a1) const;
```
a user would write:
```cpp
virtual R Baz(A1 a1) const { return gmock_Baz.Invoke(a1); }
mutable FunctionMocker<R(A1)> gmock_Baz;
```
Mocking overloaded functions is a little bit more involved. For each overloaded
version, the user needs to define an overloaded mock controller function, e.g.
```cpp
virtual R Bar(A a) { return gmock_Bar_1.Invoke(a); }
MockSpec<R(A)>& gmock_Bar(Matcher<A> a) {
return gmock_Bar_1.With(a);
}
virtual R Bar(B b, C c) { return gmock_Bar_2.Invoke(b, c); }
MockSpec<R(B, C)>& gmock_Bar(Matcher<B> b, Matcher<C> c) {
return gmock_Bar_2.With(b, c);
}
private:
FunctionMocker<R(A)> gmock_Bar_1;
FunctionMocker<R(B, C)> gmock_Bar_2;
```
If a method is overloaded on the const-ness of this object, the user can
distinguish between the two overloaded versions by using a const- vs non-const-
reference to the mock object. The `Const()` function provided by gMock can be
used to get a const reference to an object conveniently:
```cpp
template <typename T>
inline const T& Const(const T& x) { return x; }
```
### Syntax for Setting Default Actions and Expectations
For each mock function, there are two interesting properties for a user to
specify:
1. the **default action**: what the function should do by default when invoked,
and
2. the **expectations**: how the function will be called in a particular test.
While the default actions of a mock class usually don't change from test to
test, a user typically sets different expectations in different tests.
The following syntax is proposed for setting the default action of and the
expectations on a mock function:
```cpp
ON_CALL(mock-object, method(argument-matchers))
.With(multi-argument-matcher) ?
.WillByDefault(action);
```
The `ON_CALL()` statement defines what a mock function should do when its
arguments match the given matchers (unless the user overrides the behavior in
`EXPECT_CALL()`). The `With()` clause is optional. The `WillByDefault()` clause
must appear exactly once.
```cpp
EXPECT_CALL(mock-object, method(argument-matchers))
.With(multi-argument-matcher) ?
.Times(cardinality) ?
.InSequence(sequences) *
.WillOnce(action) *
.WillRepeatedly(action) ?
.RetiresOnSaturation(); ?
```
The `EXPECT_CALL()` statement says that the mock function should be called the
given number of times (`cardinality`), in the order determined by the
`sequences`, and with arguments that satisfy the given `matchers`. When it is
called, it will perform the given `action`. In this statement, all clauses are
optional and you can repeat `WillOnce()` any number of times. When no action is
specified, the default action defined by `ON_CALL()` will be taken.
For non-overloaded methods, '(argument-matchers)' may be omitted:
```cpp
ON_CALL(mock-object, method)
.With(multi-argument-matcher) ?
.WillByDefault(action);
EXPECT_CALL(mock-object, method)
.With(multi-argument-matcher) ?
cardinality and actions
```
This allows test writers to omit the parameter list and match any call to the
method. Doing so eases the burden on test maintainers when refactoring method
signatures. The 'With()' clause is still optional when the parameter list is
omitted.
We make `ON_CALL()` and `EXPECT_CALL()` macros such that we can tell the mock
object the file name and line number of a rule, which can be used to produce
better error messages at run time. When running a test inside Emacs and an
expectation is violated, the user can jump to the expectation by hitting
`<return>` on the message.
#### Argument Matchers
An `argument-matcher` can be any of the following:
```cpp
Void(), Eq(value), Ge(value), Gt(value), Le(value), Lt(value), Ne(value),
HasSubstring(string), SubstringOf(string),
Same(value), Anything(), Any<type>(), Not(argument-matcher), AnyOf(argument-matchers), AllOf(argument-matchers)
```
In addition, a user can define custom matchers by implementing the
`MatcherImplInterface<type>` interface (TBD).
#### Multi-argument Matchers
Matchers in the previous section match one argument at a time. Sometimes it's
necessary to check all arguments together. This is when multi-argument matchers
are needed:
```cpp
Eq(), Ge(), Gt(), Le(), Lt(), Ne(),
HasSubstring(), SubstringOf(),
Same(), AnyThings(), Not(multi-argument-matcher), AnyOf(multi-argument-matchers), AllOf(multi-argument-matchers)
```
When there are multiple `WithArguments()` clauses in a rule, all of them have to
be satisfied for the rule to match a call.
A user can define new multi-argument matchers by implementing the
`MatcherImplInterface<std::tuple<type1, ..., type_n> >` interface (TBD).
#### Actions
```cpp
Return(), Return(value), DoDefault(), Fail(string),
SetArgPointee<N>(value), DoAll(actions), ...
```
The version of `Return()` that takes no argument is for mocking `void`-returning
functions. The clauses are all statically typed, so a user won't be able to
mistakenly use `Return()` when the mocked function has a non-void return type,
or to use `Return(value)` when the function returns `void`.
On consecutive calls that match a given expectation, actions specified in
multiple `WillOnce()` clauses in the expectation will be used in the order they
are presented. After all `WillOnce()` clauses have been exhausted, the action
specified by `WillRepeatedly()` will always be used. If there is no
`WillRepeatedly()`, the default action defined by `ON_CALL()` will be taken.
When side effects need to be mocked (e.g. changing a field or a global variable,
calling a function of a class-typed argument, and so on), users can define a
custom action by implementing the `ActionImplInterface<return-type(type1, ...,
type-n)>` interface (TBD).
#### Cardinalities
A cardinality tells how many times a function is expected to be called. The
number doesn't have to be always exact, as we don't want to over-specify the
behavior and result in brittle tests.
```cpp
integer, AtLeast(n), AtMost(n), Between(m, n), AnyNumber()
```
This set can be extended by the user implementing the `CardinalityImplInterface`
interface (TBD).
If no cardinality is specified in an `EXPECT_CALL()` statement, gMock will infer
it this way:
* If there are n `WillOnce()` clauses but no `WillRepeatedly()`, the
cardinality is n;
* If there are n `WillOnce()` clauses and a `WillRepeatedly()`, the
cardinality is `AtLeast(n)`.
#### Sequences
Often we want to specify the order in which mock functions are called. However,
we may not want to specify a total order as that may lead to flaky tests that
will be broken by unrelated changes. For this reason, gMock allows the user to
specify a partial order on the calls by organizing them into *sequences*.
Basically, a sequence is a chain of expectations that have to happen in the
order they are defined. Sequences are identified by their names. For example,
the following defines a sequence named `"a"`, which contains two expectations
where the first has to happen before the second:
```cpp
Sequence a;
EXPECT_CALL(mock_foo, Func1(Anything()))
.Times(1)
.InSequence(a);
EXPECT_CALL(mock_bar, Func2(Eq(2)))
.Times(3)
.InSequence(a);
```
Note that expectations in the same sequence don't have to be on the same object
or same function, as the above example shows.
An expectation can belong to any number of sequences, in which case all order
constraints have to be honored. For convenience, we allow `InSequence()` to take
multiple sequences. In the following example, the first expectation must be
matched before the second and the third, but we don't care about the relative
order of the latter two:
```cpp
Sequence a, b;
EXPECT_CALL(mock_foo, Func1(Anything()))
.Times(1)
.InSequence(a, b);
EXPECT_CALL(mock_bar, Func2(Eq(2)))
.Times(AnyNumber())
.InSequence(a);
EXPECT_CALL(mock_bar, Func2(Eq(5)))
.Times(AnyNumber())
.InSequence(b);
```
For convenience, we allow an expectation to contain multiple `InSequence()`
clauses, in which case their arguments will be joined. For example, another way
to write the first expectation in the above example is:
```cpp
EXPECT_CALL(mock_foo, Func1(Anything()))
.Times(1)
.InSequence(a)
.InSequence(b);
```
A common scenario is that the user wants all expectations to match in the strict
order they are defined. Instead of letting the user put `InSequence()` in every
expectation, we provide the following short-hand:
```cpp
{
InSequence s;
EXPECT_CALL(...)...;
EXPECT_CALL(...)...;
...
}
```
In the above snippet, when the variable `s` is constructed, gMock will generate
a unique new sequence and automatically put each `EXPECT_CALL()` in the scope of
`s` into this sequence. The result is that this group of expectations must match
in the strict order.
The user can also use an existing sequence like this:
```cpp
Sequence a;
...
{
InSequence s(a);
EXPECT_CALL(...)...;
EXPECT_CALL(...)...;
...
}
```
This can be useful if an existing sequence needs to be extended.
#### Examples
```cpp
EXPECT_CALL(mock_goat, Eat(Eq(5), Anything()))
.WillOnce(Return(false));
```
The mock goat will be told to `Eat()` 5 of something exactly once; the method
should return `false`.
```cpp
EXPECT_CALL(mock_goat, Drink(HasSubstring("milk")))
.Times(1);
```
The mock goat will be told to `Drink()` something that contains milk once; the
method should perform its default action when invoked.
```cpp
EXPECT_CALL(mock_elephant, Eat(Same(mock_goat)))
.Times(0);
```
The mock elephant should never be told to `Eat()` the poor mock goat, which
would be a terrible thing.
```cpp
Sequence a;
EXPECT_CALL(mock_elephant, Eat(Anything()))
.InSequence(a)
.WillOnce(Return(true));
EXPECT_CALL(mock_elephant, Walk(Ge(5)))
.Times(AtLeast(1))
.InSequence(a)
.WillOnce(Return(2));
```
The mock elephant will be told to `Eat()` something; after that it will be told
to `Walk()` >= 5 meters at least once; the `Walk()` method should return 2 the
first time, and should do the default action in future calls.
#### Syntax Checking
We will use a combination of compile-time and run-time checks to catch syntax
errors. In particular, the spelling and types of the individual clauses will
(obviously) be done by the C++ compiler, while we'll enforce the order and
counts of the clauses via run-time checks.
Please note that technically it is possible to do the latter checks at compile
time too, and that is the approach of jMock and Mockpp. For the designer of an
embedded domain-specific language (EDSL), it is appealing to leverage the
compiler of the hosting language (C++ in this case) to parse code in the EDSL
and catch errors in it as much as possible. It is also an interesting exercise
in pushing the envelope of EDSL implementation techniques.
However, while we initially wanted to go with jMock's approach, we now think
it's better to defer such checks to run time. The reasons are:
1. Doing the checks at run time *significantly* reduces the number of template
classes and simplifies the implementation. This is not only a benefit for
the author and maintainer of gMock, but also makes it much easier for a user
to learn gMock. New and existing users will have to read the gMock header
files from time to time, so it's important to keep the public interface
small. As an example of what happens when the API is not kept small, try to
read the header files of Mockpp - you will find a plethora of template
classes that reference each other, and it's very difficult to tell what
different roles they play.
1. The jMock approach enables the IDE to automatically suggest the next clause
when a user is writing an expectation statement and thus makes it trivial to
write syntactically correct expectations. Unfortunately, such benefit is
merely theoretic for most C++ users, as C++ is such a complex language that
most IDEs do a poor job at understanding the user's source code and
suggesting auto-completion.
1. C++ templates generate horrible, horrible compiler errors that often baffle
even experienced programmers. By enforcing the syntax at compile time, we
subject gMock's users to the mercy of the C++ compiler, which will generate
lengthy and cryptic errors when a user makes a small mistake in the syntax.
It would be much better for us to generate the errors at run time, as we can
control the messages and choose plain and clear language that guides the
user to fix the problem.
1. The default action and expectation statements in gMock are *declarative*,
and typically each of them will be executed once during the test (not to be
confused with a rule *matching* an invocation multiple times). Therefore
there should be little concern that a syntax error is not caught because the
statement is never actually executed.
### Formal Semantics
The previous section presented the syntax and informally explained the meanings
of various clauses. To avoid ambiguities and make sure we all have the same
understanding on the meaning of a complete test using mock objects, we need to
define the semantics of gMock more strictly.
For an expectation rule to match an actual invocation, three types of
constraints have to be satisfied at the same time:
1. the order constraints (does the call occur in the right order?),
2. the cardinality constraints (can the rule accept more invocations?), and
3. the argument constraints (do all arguments satisfy their matchers?).
As the designer of gMock, we need to decide in which order these constraints
should be applied and how to resolve ambiguities. Our goal is to choose a
semantics that is easy to understand and allows the user to easily express
properties useful for writing tests.
Given that gMock borrows heavily from jMock, naturally one would try to adopt
jMock's semantics. I couldn't find a documentation on that unfortunately. The
following semantics is based on my reverse-engineering jMock and what I think is
reasonable. It differs from the jMock semantics in several important regards.
The exact differences and the rationale behind our decision can be found on the
c-mock-dev [archive](https://g.corp.google.com/group/c-mock-dev-archive) and are
not repeated here.
The proposed semantics can be summarized by two simple principles:
1. **The orders are sacred**: under no circumstance can an expectation in a
sequence to match before all expectations that appear earlier in the same
sequence have been satisfied; and
2. **Earlier rules take precedence:** when multiple rules can match an
invocation without violating the order constraints, the one defined the
earliest wins.
To define the semantics formally, we will use the following terminology:
* An `ON_CALL()` statement defines a **default action**.
* An `EXPECT_CALL()` statement defines an **expectation**.
* An expectation is **active** iff it still can be used to match invocations.
Otherwise it is **retired**. Initially, all expectations are active.
* An expectation X is an **immediate pre-requisite** of another expectation Y
iff there exists a sequence S where X and Y are both in S, X is defined
before Y, and there is no expectation in S between X and Y.
* An expectation X is a **pre-requisite** of another expectation Y iff there
exists a list X[0] = X, X[1], ..., X[n] = Y, where X[i] is an immediate
pre-requisite of X[i+1] for all i.
* An expectation (or its cardinality constraint) is said to be **satisfied**
iff it has reached its minimum number of allowed invocations.
* An expectation (or its cardinality constraint) is said to be **saturated**
iff it has reached its maximum number of allowed invocations. A saturated
expectation by definition must be satisfied, but not vice versa.
After the user has set the default action and the expectations, when a mock
function is called, the following algorithm (in pseudo code) will be used to
find the matching expectation and the matching action:
```cpp
void TryToDoDefault(FunctionMocker& m, const Arguments& args) {
if (m has a default action for arguments args) {
perform the default action;
} else {
raise error("No action is specified.");
}
}
void OnInvocation(FunctionMocker& m, const Arguments& args) {
for_each (active expectation e on function m in the order
the expectations are defined) {
if (all pre-requisites of e are satisfied &&
args match e's argument matchers) {
// We found a match!
if (e.is_saturated)
raise error("Invocation upper bound exceeded.");
e.invocation_count++;
retire all prerequisites of e;
if (e.retires_on_saturation && e.is_saturated)
e.is_active = false;
if (e has more action left) {
a = e.get_next_action();
perform a;
} else {
TryToDoDefault(m, args);
}
return;
}
}
TryToDoDefault(m, args);
}
```
To find the default action for the given arguments, we look through all
`ON_CALL()` rules for the mock function, and pick the first one where all
argument matchers are satisfied, if any.
Since C++ exceptions are disabled in `google3`, **we will abort the current
process when gMock raises an error**. We cannot just return from the current
function like what gUnit does, as the mock functions will be called from the
production code under test, and we don't have the luxury to change the
production code at each call site to propagate the error. This is unfortunate,
but I don't see a better solution without enabling exceptions.
The real implementation will be more sophisticated in order to get a decent
performance (e.g. we'll memoize and use other tricks), but the end result must
match the above reference implementation.
**Note:** If you carefully inspect the algorithm, you should convince yourself
that an expectation whose cardinality is `0` has no effect whatsoever, as it is
always satisfied and saturated. This means that you can write such an
expectation, but it won't affect your test in any way. Indeed, this is jMock's
behavior, and jMock's documentation suggests to use `Never()` (jMock's
equivalent of the `0` cardinality) for documentation purpose only.
This bothers me as it contradicts with what one would naturally expect. When I
see
```cpp
EXPECT_CALL(mock_foo, Bar(Eq(5)))
.Times(0);
```
I would think that it will be an error if `mock_foo.Bar(5)` is ever called, and
gMock will catch this error at run time. However, jMock tells me that it will
not try to enforce this, and I should treat the statement as if it doesn't
exist.
I propose to give `Times(0)` a semantics that I think is more intuitive. Namely,
we should treat
```cpp
EXPECT_CALL(mock-object, method(argument-matchers))
.WithArguments(multi-argument-matcher)
.Times(0)
.InSequence(sequences);
```
as if it were
```cpp
EXPECT_CALL(mock-object, method(argument-matchers))
.WithArguments(multi-argument-matcher)
.Times(AnyNumber())
.InSequence(sequences)
.WillOnce(Fail("Unexpected call."));
```
I don't like making this a special case, but the other choice seems worse.
**Note:** If a call doesn't match any explicitly written `EXPECT_CALL()`
statement, gMock will perform the default action (as long as it exists) instead
of raising an "unexpected call" error. If you want to assert that a function
should never be called, you must explicitly write an `EXPECT_CALL()` with a `0`
cardinality. This design is picked to allow modular tests:
An interface may contain many methods. Typically, each test will be interested
in only a small number of them, as we favor small and focused tests. Such a test
shouldn't start to fail when the code under test is modified to call functions
not interesting to the test. If no `EXPECT_CALL()` were to mean "no call is
allowed", we would have to say
```cpp
EXPECT_CALL(mock_foo, UninterestingMethod(Anything()))
.Times(AnyNumber());
```
for **every** method we do **not** care about. It can be very tedious. Worse,
when we add methods to the interface or remove methods from it, we have to
update every existing test. Clearly this isn't modular and won't scale.
#### Examples
If you are not interested in whether a function will be called or not, you just
don't say anything about it. If the function is called, its default action will
be performed.
If you want to make sure that a function is never called, say it explicitly:
```cpp
EXPECT_CALL(mock_foo, Bar).Times(0);
// or:
EXPECT_CALL(mock_foo, Bar(Anything())).Times(0);
```
If you expect certain calls to a function and want to ignore the rest, just
specify the calls you are explicitly expecting:
```cpp
EXPECT_CALL(mock_foo, Bar(Eq(1)))
.WillOnce(Return(2));
EXPECT_CALL(mock_foo, Bar(Eq(2)))
.Times(AtMost(5))
.WillRepeatedly(Return(3));
```
If you expect certain calls to a function and don't want to allow any other
calls to it, just add a `Times(0)` expectation **after** the normal
expectations:
```cpp
EXPECT_CALL(mock_foo, Bar(Eq(1)))
.WillOnce(Return(2));
EXPECT_CALL(mock_foo, Bar(Eq(2)))
.Times(AtMost(5))
.WillRepeatedly(Return(3));
// Any call to mock_foo.Bar() that doesn't match the above rules
// will be an error.
EXPECT_CALL(mock_foo, Bar(Anything()))
.Times(0);
```
Here's one complete example:
```cpp
ON_CALL(mock_foo, Bar(Anything()))
.WillByDefault(Return(1));
Sequence x;
EXPECT_CALL(mock_foo, Bar(Ne('a'))) // Expectation #1
.InSequence(x)
.WillOnce(Return(2))
.WillRepeatedly(Return(3));
EXPECT_CALL(mock_foo, Bar(Anything())) // Expectation #2
.Times(AnyNumber())
.InSequence(x);
mock_foo.Bar('b'); // Matches expectation #1; returns 2.
mock_foo.Bar('c'); // Matches expectation #1; returns 3.
mock_foo.Bar('b'); // Matches expectation #1; returns 3.
mock_foo.Bar('a'); // Matches expectation #2; returns 1.
// Now that expectation #2 has been used, expectation #1 becomes
// inactive (remember that the order is sacred), even though it's not
// yet saturated.
mock_foo.Bar('b'); // Matches expectation #2, returns 1.
```
Another one:
```cpp
Sequence a, b;
EXPECT_CALL(mock_foo, Func1(Void())) // #1
.Times(1)
.InSequence(a);
EXPECT_CALL(mock_bar, Func2(Anything()) // #2
.Times(AtLeast(1))
.InSequence( b);
EXPECT_CALL(mock_foo, Func3(Eq(0))) // #3
.Times(AtMost(2))
.InSequence(a, b);
EXPECT_CALL(mock_foo, Func3(Anything())) // #4
.InSequence(a);
// The order constraints can be illustrated as
//
// #1 < #3 < #4
// #2 < #3
mock_foo.Func1(); // Matches #1
// Now #1 is saturated but not retired.
// If Func1() is called again here, it will be an upper-bound error.
// It would be an error to call mock_foo.Func3(0) here, as #2 is its
// pre-requisite and hasn't been satisfied.
mock_bar.Func2(1); // Matches #2, which is now satisfied.
mock_foo.Func3(1);
// Matches #4. This causes all of #4's remaining pre-requisites (#2
// and #3) to become inactive. Note that #3 is trivially satisfied
// as that AtMost(2) doesn't require it to match any invocation.
```
Yet another one:
```cpp
EXPECT_CALL(mock_foo, Func(Eq(1))) // #1
.WillOnce(Return(2))
.RetiresOnSaturation();
EXPECT_CALL(mock_foo, Func(Anything())) // #2
.WillOnce(Return(3));
mock_foo.Func(1); // Matches #1.
// Now #1 is satisfied, saturated, and retired.
mock_foo.Func(1); // Matches #2.
// Since #1 is retired now, it doesn't participate in matching function
// calls. Otherwise this would cause an upper-bound-exceeded failure.
```
### Verifying that All Expectations Are Satisfied
During a test, gMock will verify that each invocation to a mock function matches
one of the expectation rules. However, at the end of a test, we will want to
verify that *all expectations for the mock function have been satisfied*. This
is done by the destructor of the `FunctionMocker<...>` class:
### Life of a Mock
Now let's put the pieces together and see the complete process of using mock
objects in a test. Typically, the user should do it in the following order:
* **C:** *Constructs* the mock objects;
* **B:** Set their default **behaviors** using `ON_CALL()`;
* **E:** Set **expectations** on them using `EXPECT_CALL()`;
* **I:** Exercise the production code, which will **invoke** methods of the
mock objects;
* **D:** *Destructs* the mock objects, which will cause gMock to verify that
all expectations are satisfied.
Usually, the user can do step 1 and 2 during the set-up phase of the test, step
3 and 4 in the test function body, and step 5 in the tear-down phase.
If the user performs a step out of sequence (e.g. an `EXPECT_CALL()` is
encountered after the mock function is already been called by the test and
before `Verify()` is called), the behavior is **undefined**. gMock will try to
print a friendly error message, but doesn't guarantee to catch all possible
violations, and the initial version may not implement this error check at all.
Valid sequences of using a mock object can be described using the regular
expression
```none
CB*E*I*D
```
### Typing of Argument Matchers
Argument matchers in gMock are statically typed. If we don't provide automatic
conversion between matchers of "compatible" types, the user experience will be
rather unpleasant. Covariance and contravariance are two common schemes for
imposing a sub-typing relation between types. Our observation is that neither
works for matchers in general, and gMock must leave the decision to individual
matcher authors.
#### Background: How Argument Matchers Work
In gMock, argument matchers are used to determine if the actual arguments in a
function invocation match the test writer's expectation. Conceptually, a matcher
for arguments of type `T` implements the following interface:
```cpp
class Matcher<T> {
virtual bool Matches(T x) = 0;
};
```
For a method with argument type `T`:
```cpp
virtual void Func(T x);
```
its mock will be declared as something like (the actual declaration will be more
complicated but the idea remains the same):
```cpp
void Func(Matcher<T>* x);
```
When the mock `Func()` is invoked with an argument value `v`, which has type
`T`, `Matches(v)` will be called to determine if it's a match.
#### Need for Sub-typing
A straightforward way to mock a method with parameter types `T1`, `T2`, ..., and
`Tn` is to use a list of matchers of type `Matcher<T1>`, `Matcher<T2>`, ..., and
`Matcher<Tn>`. However, this simplistic approach has a usability problem.
Suppose we have a series of functions and their mocks:
```cpp
void Func1(char a);
void Func1(Matcher<char>* a);
void Func2(const char a);
void Func2(Matcher<const char>* a);
void Func3(char& a);
void Func3(Matcher<char&>* a);
void Func4(const char& a);
void Func4(Matcher<const char&>* a);
void Func5(char* a);
void Func5(Matcher<char*>* a);
void Func6(const char* a);
void Func6(Matcher<const char*>* a);
```
(note that `Func2()` has a `const` parameter. Since argument matchers are not
allowed to modify the arguments in any case, technically we could use a
`Matcher<char>` in the mock of `Func2()`. However, we want to make sure that a
user can define the mock using a `Matcher<const char>` too, as this makes the
task of the mock class generator easier.) and some simple, pre-defined matcher
factories:
```cpp
// Matches if the argument equals the given value x.
Matcher<T>* Eq(T x);
// Matches if the argument has the same identify as the
// given object x.
Matcher<T&>* Same(T& x);
```
then a user might be surprised when trying to use these mocks:
```cpp
Func1('a'); // Invokes the real method. This works fine.
Func1(Eq('a')); // Invokes the mock method. This works fine too.
Func2('a'); // Invokes the real method. This works fine.
Func2(Eq('a')); // Compiler ERROR - surprise!!! Why can't I say that
// the argument, which is a const char, should be equal
// to 'a'?
char a = 'a';
Func3(a); // Invokes the real method. This works fine.
Func3(Same(a)); // Fine. The argument should reference the variable a.
Func3(Eq('a')); // Compiler ERROR - surprise!!! Why can't I say that
// the argument, which is a char&, should have a value
// 'a', which is a char?
const char b = 'b';
Func4(b); // Fine.
Func4(Same(b)); // Fine. The argument should reference the variable b.
Func4(Eq(b)); // Compiler ERROR - surprise!!! Why can't I say that
// the argument, which is a const char&, should have
// a value equal to b, which is a const char?
Func4(Same(a)); // Compiler ERROR - surprise!!! Why can't I say that
// the argument, which is a const char&, should reference
// a, which is a char?
char* p = NULL;
Func5(p); // Fine.
Func5(Eq(p)); // Fine.
Func6("a"); // Fine.
Func6(Eq("a")); // Fine.
Func6(Eq(p)); // Compiler ERROR - surprise!!! Why can't I say that
// the argument, which is a const char*, should point
// to where p, which is a char*, points to?
```
(In Java, this issue isn't nearly as severe, as Java has neither reference nor
`const`. Lucky them.)
The compiler errors can be worked around using explicit template instantiating
in most cases, but require more dirty hacks in some others:
```cpp
// The following works:
Func2(Eq<const char>('a'));
Func4(Eq<const char&>(b));
Func4(Same<const char>(a));
Func6(Eq<const char*>(p));
// but this doesn't:
Func3(Eq<char&>('a')); // Compiler ERROR!
// You have to use a temporary variable, and pray that it's not
// accidentally changed before the actual invocation occurs.
// No, you cannot make the temporary variable const - that won't
// compile.
char temp = 'a';
Func3(Eq<char&>(temp));
```
Having to use these tricks all the time is a big bummer and makes the tests
harder to read. The author of Mockpp either chose to ignore this problem, or
wasn't aware of it, but I don't think that's a good solution.
To give the user a satisfying experience, I believe we should provide automatic
conversions between matchers of "compatible" types when it makes sense (i.e. we
should establish a sub-typing relation between matcher types). The challenges
are:
1. Deciding when "it makes sense",
1. Making sure the conversions are neither too restricted nor too liberal,
1. Making it possible for the user to understand the compiler errors when
automatic conversions cannot be done,
1. Making the rules easy to learn and remember, and
1. Implementing it.
#### Covariant or Contravariant?
We already know that making the matchers **invariant** (i.e. no auto-conversion
between matcher types) doesn't work, but what about **covariant** (`Matcher<A>`
can be used as `Matcher<B>` as long as `A` can be used as `B`) or
**contravariant** (`Matcher<A>` can be used as `Matcher<B>` as long as `B` can
be used as `A`)? Would one of them work?
It's easy to see that covariance doesn't work in general, as it requires a
matcher expecting a sub-type value to inspect a super-type value. What if the
matcher needs to look at a field that's only present in the sub-type?
On the surface level, it seems that contravariance is what we need: if type `B`
can be implicitly converted to type `A`, then given an argument of type `B`, we
can convert it to type `A` and then ask a `Matcher<A>` whether the converted
value matches. This means that we can use a `Matcher<A>` in place of a
`Matcher<B>`.
For example, given a class hierarchy and some (real and mock) functions that use
the classes:
```cpp
class Father { ... };
class Son : public Father {
public:
...
// New method not present in Father:
virtual bool PropertyFoo() const;
};
class Grandson : public Son { ... };
void InviteFather(Father* obj);
void InviteFather(Matcher<Father*>* m);
void InviteGrandson(Grandson* obj);
void InviteGrandson(Matcher<Grandson*>* m);
```
we can expect to write the following:
```cpp
// Matches if the argument's PropertyFoo() method returns true.
Matcher<Son*>* HasFoo() { ... }
// The compiler should reject this as a Father object doesn't have
// the PropertyFoo() method:
//
// InviteFather(HasFoo());
// This should work, as a Grandson object is also a Son object and
// has the PropertyFoo() method.
InviteGrandson(HasFoo());
```
In the latter case, the actual argument (of type `Grandson*`) will be implicitly
converted to a `Son*` and then passed to the matcher.
However, things are not always so simple. Take the example of the equality
matcher, `Func5()`, and `Func6()` we saw earlier:
```cpp
// Matches if the argument equals the given value x.
Matcher<T>* Eq(T x);
void Func5(char* a);
void Func5(Matcher<char*>* a);
void Func6(const char* a);
void Func6(Matcher<const char*>* a);
```
By making matchers contravariant, we have
```cpp
// Remember that a char* can be implicitly converted to a const char*.
Func5(Eq("a")); // Compiles, but we DON'T want it to!!! The user is
// trying to say that the actual argument (a char*)
// can be "a", and we should catch this error.
char* p = ...;
Func6(Eq(p)); // Still a compiler ERROR, as a const char* cannot be
// implicitly converted to a char*, which Eq(p) expects.
```
Clearly this isn't what we want. In fact, we want `Eq(value)` to be covariant:
```cpp
char* p = ...;
Func5(p);
Func5(Eq(p));
// Func5("a"); // The compiler will reject this,
// Func5(Eq("a")); // and thus should reject this too.
Func6("a");
Func6(Eq("a"));
Func6(p); // The compiler accepts this,
Func6(Eq(p)); // and thus should accept this too.
```
In another example:
```cpp
void InviteSon(Son* obj);
void InviteSon(Matcher<Son*> m);
Father* father = ...;
Grandson* grandson = ...;
InviteSon(grandson); // The compiler accepts this,
InviteSon(Eq(grandson)); // and thus should accept this too.
// InviteSon(father); // The compiler will reject this,
// InviteSon(Eq(father)); // and thus should reject this too.
```
So, what was the problem? The key insight here is that *one size doesn't fit
all*. While some matchers should be contravariant (like `HasFoo()`), some should
be covariant (like `Eq(value)` and `Same(object)`). *The decision has to be left
to the individual matcher authors. gMock should not impose a global policy on
all the matchers.*
#### Implementing Automatic Type Conversion
In C++, you have several options if you want to make one class `A` act like
another class `B`:
1. Derive `A` from `B`;
1. In class `B`, define a public single-parameter constructor `B::B(const A&)`
(don't make it `explicit`);
1. In class `A`, define a type-conversion operator for type `B`.
Each of these approaches has its limitations:
* #1 is most straightforward and requires the least amount of work. However,
it means that an `A` object will contain all the members of `B`. It may not
work for you if you want to be able to auto-convert `A` to multiple classes,
and it certainly won't work if you want to convert `A` to an infinite number
of other classes.
* #2 requires you to be able to edit the implementation of `B`. This is not
always possible, for example, when `B` is a class defined in a standard
library and you are a user of that library. It's a closed approach, meaning
that only the owner of `B` can decide which classes can be converted to `B`.
* #3 requires more work to implement typically, but doesn't have the problems
of #1 and #2. In particular, you can define a template type-conversion
operator to convert `A` to an infinite number of other classes of your
choice.
Also, remember that the compiler will only automatically insert one level of
type conversion on your behalf. For example, if `A` can be implicitly converted
to `B`, which can be implicitly converted to `C`, and you have a function
expecting a `C`, you cannot give the function an `A` without explicit casting,
unless `A` can be implicitly converted to `C` too.
gMock defines the `Matcher<T>` interface, which a user cannot modify. When
defining a polymorphic matcher (e.g. `Eq(value)`), a user needs to make it
behave like a (potentially infinite) number of matchers of different types. This
means that the last implementation technique should be used.
### Comparison with Mockpp and jMock
See GMockVsMockppAndJMock.
## Project
This design doc describes the project "[gMock](http://p?p=6164) - a framework
for writing C++ mock classes" in PDB.
## Code Location
This is a new project, so no existing file is expected to be touched. The
implementation is added in directory `//depot/google3/testing/base`.
## Group Members
ZhanyongWan : spending 60% of his time on this.
## Caveats
We considered existing solutions, but don't think they would work well enough
for Google. For details, please refer to MockppStudy.
TODO:
* Explain why we pick the EDSL-style syntax.
## Documentation Plan
In additional to this design doc, a user's guide, an FAQ, and a codelab will
also be written.
## Testing Plan
gMock will be thoroughly tested on Linux using gUnit.
## Work Estimates
* **Inspecting existing solutions:** The goal is to understand how other
people have approached this problem, what they did well, and what did
poorly. In addition to studying solutions for C++ (mockpp), we will also
study solutions for Java (jMock and EasyMock). **3 weeks.**
* **Initial design and prototyping:** Come up with a design tailored to C++'s
specifics and Google's unique requirements, taking into account lessons
learned from other solutions. **3 weeks.**
* **Soliciting feedback on the design:** Listen to `testing-technology`,
`testing-grouplet`, `c-users`, `c-mock-dev`, and `gunit-users`; revise the
design based on the feedback. **3 weeks.**
* **Initial implementation:** **6 weeks.**
* **Documentation:** Write the user's guide and a codelab. **3 weeks.**
* **Company-wide roll-out:** Implement a process for promoting and tracking
adoption. **1 week.**
* **Customer support, incremental improvements, and maintenance:** **On-going
effort.**
## Potential Patents
We'll know whether there will be patentable inventions when we have a more
concrete design and prototype. At that time, we should talk to Google's
[patent counsel](mailto:patents@google.com).
## Things that Don't Apply
### Security Considerations
This is an internal library for writing (unit) tests, and will not be used in
production. Therefore there is no security concern.
### Privacy Considerations
gMock will not touch any user data. Therefore there is no concern about user
privacy.
### Standards
There is no existing standard in creating C++ mocks.
### Logging Plan
This is an internal library and will not handle any user request. Therefore
there is no plan for logging.
### Monitoring Plan
This is an internal library and will not run on our production servers.
Therefore no monitoring is required.
### Internationalization Plan
gMock is not visible to external users, so there is no plan to internationalize
it.
### Billing Plan and Tax Plan
gMock is an internal library and doesn't involve money, so there is no billing
plan or tax plan.
### Launch Plans
gMock will not launch externally as a Google property. However, we may later
decide to open source it.
## References
* [jMock 1 cheat sheet](https://wiki.corp.google.com/twiki/bin/view/Main/JMockOneCheatSheet) -
I compiled this from the downloaded jMock 1 source code.
* [jMock 1 JavaDoc](http://www.corp.google.com/~wan/jmock1/javadoc/) - I built
this locally. The one on http://jmock.org can be slow and may change.
* [jMock 2 cheat sheet](http://www.jmock.org/cheat-sheet.html) - as found on
http://jmock.org.
* [jMock 2 cheat sheet](https://wiki.corp.google.com/twiki/bin/view/Main/JMockTwoCheatSheet) -
I compiled this from the jMock 2 source code in CVS as of 3/21.
* [jMock 2 JavaDoc](http://www.corp.google.com/~wan/jmock2/javadoc) - I
generated this locally from the jMock 2 source code in CVS as of 3/21. No
official jMock 2 JavaDoc is available yet, as the library hasn't been
released.
* [mockpp cheat sheet](https://wiki.corp.google.com/twiki/bin/view/Main/MockppCheatSheet) -
I compiled this from the mockpp source code.
* [mockpp API docs](http://mockpp.sourceforge.net/api-doc/index.html) -
external.
## Acknowledgments
This design doc contains many ideas from PiotrKaminski. We'd also like to thank
the following people for contributing ideas to this project:
JoeWalnes, CraigSilverstein, JeffreyYasskin, KeithRay, MikeBland.
# googletest gMock Users Guide
go/gmockguide
Welcome to googletest: Google's C++ testing and mocking framework. gMock is a
mocking part of googletest.
* [OSS Version](https://github.com/google/googletest)
* [Google3](http://google3/third_party/googletest/)
* If you are new to gMock, start with [*gMock for Dummies*](for_dummies.md) to
learn the basic usage.
* Read [gMock Cookbook](cook_book.md) to learn more advanced usage and useful
tips.
* For a quick reference, check out [gMock Cheat Sheet](cheat_sheet.md).
* If you have questions, search [gMock FAQ](#GMockFaq) and the gmock-users@
archive before sending them to gmock-users@.
<!-- GOOGLETEST_CM0035 DO NOT DELETE -->
<!--#include file="for_dummies.md"-->
#### Side Effects
<!-- mdformat off(github rendering does not support multiline tables) -->
| Matcher | Description |
| :--------------------------------- | :-------------------------------------- |
| `Assign(&variable, value)` | Assign `value` to variable. |
| `DeleteArg<N>()` | Delete the `N`-th (0-based) argument, which must be a pointer. |
| `SaveArg<N>(pointer)` | Save the `N`-th (0-based) argument to `*pointer`. |
| `SaveArgPointee<N>(pointer)` | Save the value pointed to by the `N`-th (0-based) argument to `*pointer`. |
| `SetArgReferee<N>(value)` | Assign `value` to the variable referenced by the `N`-th (0-based) argument. |
| `SetArgPointee<N>(value)` | Assign `value` to the variable pointed by the `N`-th (0-based) argument. |
| `SetArgumentPointee<N>(value)` | Same as `SetArgPointee<N>(value)`. Deprecated. Will be removed in v1.7.0. |
| `SetArrayArgument<N>(first, last)` | Copies the elements in source range [`first`, `last`) to the array pointed to by the `N`-th (0-based) argument, which can be either a pointer or an iterator. The action does not take ownership of the elements in the source range. |
| `SetErrnoAndReturn(error, value)` | Set `errno` to `error` and return `value`. |
| `Throw(exception)` | Throws the given exception, which can be any copyable value. Available since v1.1.0. |
<!-- mdformat on -->
* When compiling with exceptions in google3, it's not enough to specify
`-fexceptions` to copts in your cc_test target. That flag will not be
inherited by gmock, and various headers will be compiled both with and
without `-fexceptions` causing subtle bugs. Instead you must pass
`--copt=-fexceptions` to the blaze command so the flag gets passed to all
targets... but this is Google and we don't use exceptions so it shouldn't
really be an issue.
# googletest Home
go/gmock
Googletest is Google's C++ testing and mocking framework. Please note that there
are legacy names you may encounter "gUnit" and "gMock" - these names are now
merged into "googletest"
<!-- GOOGLETEST_CM0035 DO NOT DELETE -->
## Testimonials
> "I'm really enjoying trying it, and it's amazing to me how far you've taken
> this in C++. It's changed the way I program (and therefore changed my life ;),
> and one of my teams has adopted it for all/most tests (and I'm working on the
> other)." \
> -- *Derek Thomson*, Google Mountain View
<section></section>
> "I started using mocks with EasyMock in Java a few years ago and found them
> **invaluable** for making unit testing as painless and effective as possible.
> I'm very glad (and amazed) to see you've managed to create something similar
> for C++. It's making the transition much more pleasant." \
> -- *David Harkness*, Google Mountain View
<section></section>
> "I #included `gmock.h` and lived to tell the tale... Kept me from having to
> depend on `MockBigtable` thus far, which is **huge**." \
> -- *Matthew Simmons*, Google NYC
<section></section>
> "I like the approach of `EXPECT_CALL` much more than EasyMock's mock modes
> (record, replay). It's the best way to ensure the user will never forget to
> verify the expectations: do it automatically!" \
> -- *Tiago Silverira*, Google Brazil
<section></section>
> "It's by far the best mocking library for C++, by a long-shot." \
> -- *Joe Walnes*, co-creator of jMock, Google London
## Learning googletest mocking
Please see the [*googletest Users Guide*](guide.md) for the combined gMock
mocking documentation.
## Resources for Users
* More docs:
* [Interview with gMock's Creator](http://www.corp.google.com/eng/testing/codegreen/v10/gMock.htm)
on the
[Feb 2008](http://www.corp.google.com/eng/testing/codegreen/v10/index.htm)
issue of [Code Green](http://go/codegreen) - discusses gMock's history
and philosophy.
* "Mockers of the (C++) world, delight!": TotT
[episode 66](http://big.corp.google.com/~jmcmaster/testing/2007/12/episode-68-mockers-of-c-world-delight.html) -
quick intro on gMock's benefits and usage
* "Mock logs better than gold": TotT
[episode 76](http://big.corp.google.com/~jmcmaster/testing/2008/02/episode-76-mock-logs-better-than-gold_21.html) -
how to test LOGs using gMock
* "Testing legacy code gently": TotT
[episode 84](http://big.corp.google.com/~jmcmaster/testing/2008/04/episode-84-testing-legacy-code-gently.html) -
using mock callbacks to test legacy code without a big refactoring
* "Literate testing with matchers": TotT
[episode 135](http://big.corp.google.com/~jmcmaster/testing/2009/06/episode-135-literate-testing-with_08.html) -
using matchers to get readable test code and readable test messages
* "Making a perfect matcher": TotT
[episode 139](http://big.corp.google.com/~jmcmaster/testing/2009/07/episode-139-making-perfect-matcher.html) -
defining custom matchers easily
* Talks
* "Declarative C++ Testing Using DSLs" talk (6/4/2008):
[abstract](https://wiki.corp.google.com/twiki/bin/view/Main/WanTalks#Declarative_C_Testing_Using_DSLs),
[slides](http://wiki.corp.google.com/twiki/pub/Main/WanTalks/0806-declarative-cpp-testing.xul#Eva)
(requires Firefox) - gMock's design and implementation tricks
* "Mocks made easy in C++ and Java" talk (4/23/2008):
[slides](http://go/MockTalk),
[fish](http://fish.corp.google.com/talks/8729/)
* "C++ mocks made easy - an introduction to gMock" talk (1/22/2008)):
[slides](http://wiki.corp.google.com/twiki/pub/Main/WanTalks/0801-mv-gmock.xul#eva)
(requires Firefox),
[video](https://video.google.com/a/google.com/?AuthEventSource=SSO#/Play/contentId=bd07003d4193a646)
* "A preview to gMock" talk (6/28/2007):
[PowerPoint slides](http://wiki.corp.google.com/twiki/pub/Main/WanTalks/0706-beijing-gmock-preview.ppt)
* Tools
* `/google/src/head/depot/google3/third_party/googletest/googlemock/scripts/generator/gmock_gen.py
*your.h ClassNames*` generates mocks for the given base classes (if no
class name is given, all classes in the file are emitted).
* Mocks
* [mock-log.h](http://s/?fileprint=//depot/google3/testing/base/public/mock-log.h) -
a sample on using gMock to create a mock class
* [gmock-sample-mock-log.cc](http://s/?fileprint=//depot/google3/testing/base/internal/gmock-sample-mock-log.cc) -
a sample on using gMock to test LOG()s
* Folks
* Meet the
[users](http://piano.kir.corp.google.com:8080/lica/?e=use%3Agmock).
* `gmock-users` list:
[subscribe](https://groups.google.com/a/google.com/group/gmock-users/topics),
[archive](https://groups.google.com/a/google.com/group/gmock-users/topics),
[smile!](http://piano.kir.corp.google.com:8080/lica/?e=gmock-users) Send
questions here if you still need help after consulting the on-line docs.
* `gmock-announce` list:
[subscribe](https://groups.google.com/a/google.com/group/gmock-announce/topics)
to this instead of `gmock-users` if you are interested in announcements
only.
## Resources for Contributors
* [Dashboard](http://unittest.corp.google.com/project/gunit-gmock/)
* [*gMock Design*](design.md) (go/gmockdesign) - the design doc
* `c-mock-dev` list (deprecated) -
[old archive](https://mailman.corp.google.com/pipermail/c/c-mock-dev/),
[new archive](https://g.corp.google.com/group/c-mock-dev-archive)
* `opensource-gmock` list - discussions on the development of gMock:
[subscribe](https://groups.google.com/a/google.com/group/opensource-gmock/subscribe),
[archive](https://g.corp.google.com/group/opensource-gmock-archive),
[smile!](http://piano.kir.corp.google.com:8080/lica/?e=opensource-gmock)
## Acknowledgments
We'd like to thank the following people for their contribution to gMock: Piotr
Kaminski, Jeffrey Yasskin (who/jyasskin), Joe Walnes, Bradford Cross, Keith Ray,
Craig Silverstein, Matthew Simmons (who/simmonmt), Hal Burch (who/hburch), Russ
Rufer, Rushabh Doshi (who/rdoshi), Gene Volovich (who/genev), Mike Bland, Neal
Norwitz (who/nnorwitz), Mark Zuber, Vadim Berman (who/vadimb).
# GMock
<!-- GOOGLETEST_CM0035 DO NOT DELETE -->
## What is gMock?
gMock is Google's framework for creating and using C++ mock classes. It helps
you design better systems and write better tests. A mock object is an object
that you use in a test instead of a real object. A mock object implements the
same interface as a real object but lets you specify at run time how the object
will be used. When you write tests that use a mock, you define expectations
about how the mock's methods will be called. Your test then verifies how your
real code behaves when interacting with the mock. See the
[Mock Objects Best Practices Guide](http://go/mock-objects#mocks-stubs-fakes)
for a comparison of mocks with stubs, fakes, and other kinds of test doubles.
For example, gMock provides a simple syntax for declaring "I expect the
RetryQuery method on this mock object to be called three times in the course of
this test". Your test will fail if the expectation isn't met.
The gMock library provides a mock framework for C++ similar to jMock or
EasyMock[?](http://go/easymock-codelab) for Java. In gMock you use macros to
define methods for your mock objects and set expectations for those methods.
gMock runs on Linux, Windows, and Mac OS X.
## What is gMock good for?
Mocks in general are good for:
- prototyping and designing new code and APIs.
- removing unnecessary, expensive, or unreliable dependencies from your tests.
gMock in particular is good for writing quality C++ mocks. Without the help of a
mocking framework like gMock, good C++ mocks are hard to create.
## What is gMock NOT good for?
gMock is not good for testing the behavior of dependencies. The point of testing
with mocks is to test the classes that use the mocks, not the mocks themselves.
Objects that have working toy implementations are called fakes instead of mocks.
For example, you could use an in-memory file system to fake disk operations.
Mocks aren't useful for very simple classes like
[Dumb Data Objects](http://big.corp.google.com/~jmcmaster/testing/2011/04/episode-220-blast-from-tott-past-dont.html).
If it's more trouble to use a mock than the real class, just use the real class.
## Who uses gMock?
There are over 30K tests using gmock. Virtually every C++ test at Google that
needs a mock object uses gMock.
## Practical matters
gMock is bundled with [gUnit](/third_party/googletest/googletest/docs/). To use
gMock,
[include a dependency](/third_party/googletest/googletest/docs/howto_cpp#LinuxTarget)
on `//testing/base/public:gunit` in the BUILD rule for your mocks, and use the
following include statement in the file that defines your mock class:
```
#include "gmock/gmock.h"
```
&nbsp; | &nbsp;
--------------------------- | ------------------------------------------
**Implementation language** | C++
**Code location** | google3/third_party/googletest/googlemock/
**Build target** | //testing/base/public:gunit
## Best practices
Use [dependency injection](http://en.wikipedia.org/wiki/Dependency_injection) to
enable easy mocking. If you define dependencies as interfaces rather than
concrete classes, you can swap out the production version of a class for a mock
during testing.
You can also use gMock during the design phase for your system. By sketching
your architecture using mocks rather than full implementations, you can evolve
your design more quickly.
## History and evolution
In January 2007 Zhanyong Wan and the Testing Technology team met with
experienced C++ engineers to find out about C++ testing needs. The team learned
that creating mocks in C++ was a major pain point. They looked around for
existing frameworks but didn't find anything satisfactory. So Zhanyong Wan
tackled the problem of creating a usable C++ mocking framework.
C++ posed a unique problem for mocking: while
[reflection](http://en.wikipedia.org/wiki/Reflection_\(computer_programming\))
in Java and Python make it easy to generate a mock implementation of any
interface, C++ does not have reflection. Wan hit on macros as a way to simplify
mock writing in C++, and gMock was born.
## Who to contact
- g/gmock-users
- g/gmock-announce
## Additional resources
- [gMock](http://go/gmock) - homepage
- [GMock for Dummies](http://<!-- GOOGLETEST_CM0013 DO NOT DELETE -->) - gets you started with gMock
quickly
- [GMock Cookbook](http://<!-- GOOGLETEST_CM0012 DO NOT DELETE -->) - recipes for common scenarios; covers
advanced usage.
- [GMock Cheat Sheet](http://<!-- GOOGLETEST_CM0020 DO NOT DELETE -->) - a quick reference
- [GMock FAQ](http://<!-- GOOGLETEST_CM0021 DO NOT DELETE -->) - frequently asked questions
- [gUnit GDH page](http://go/gunit-overview)
- [gUnit User's Guide](http://goto.corp.google.com/gunit) - gets you started
with gUnit, which is closely related to gMock
Googletest Mocking (gMock)
* [Home](index.md)
* [Overview](overview.md)
* [User's Guide](guide.md)
* [gMock For Dummies](for_dummies.md)
* [gMock Cookbook](cook_book.md)
* [gMock Cheat Sheet](cheat_sheet.md)
* [Design](design.md)
* [How To Contribute](contribute.md)
* [gMock FAQ](gmock_faq.md)
**WARNING:** This document was recently migrated from
[Goowiki](http://wtf/goowiki) (b/35424903) and may still require additional
updates or formatting. You can still access the original document on Goowiki
until the cleanup is complete:
## BUILD Rule
Add *one* of the following to your `deps`:
```build
"//testing/base/public:gunit",
"//testing/base/public:gtest_main",
```
Add this to your `.cc` file:
```cpp
#include "gmock/gmock.h"
```
Unless noted, *all functions and classes* are defined in the `::testing`
namespace.
### absl::Status
In namespace `testing::status`:
<a name="table22"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `IsOk()` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` whose status is OK. </td>
</tr>
<tr>
<td> `IsOkAndHolds(m)` </td>
<td>
`argument` is an `absl::StatusOr<T>` whose status is OK and whose inner value matches matcher `m`.
See also [`ASSERT_OK_AND_ASSIGN`](http://google3/testing/base/public/gmock_utils/status-matchers.h?q=symbol:ASSERT_OK_AND_ASSIGN).
</td>
</tr>
<tr>
<td> `StatusHasPayload()` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` whose status is not OK and which has any payload. </td>
</tr>
<tr>
<td> `StatusHasPayload<ProtoType>()` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` whose status is not OK and which has a payload of type `ProtoType`. </td>
</tr>
<tr>
<td> `StatusHasPayload<ProtoType>(m)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` whose status is not OK and which has a payload of type `ProtoType` that matches `m`. </td>
</tr>
<tr>
<td> `StatusIs(s, c, m)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` where: the error space matches `s`, the status code matches `c`, and the error message matches `m`. </td>
</tr>
<tr>
<td> `StatusIs(c, m)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` where: the error space is canonical, the status code matches `c`, and the error message matches `m`. </td>
</tr>
<tr>
<td> `StatusIs(c)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` where: the error space is canonical, and the status code matches `c`. </td>
</tr>
<tr>
<td> `CanonicalStatusIs(c, m)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` where: the canonical status code matches `c`, and the error message matches `m`. </td>
</tr>
<tr>
<td> `CanonicalStatusIs(c)` </td>
<td> `argument` is a `absl::Status` or `absl::StatusOr<T>` where: the canonical status code matches `c`. </td>
</tr>
</table>
The two- and one-argument version of StatusIs use util::GetErrorSpaceForEnum to
determine the error space. If the error code matcher is not an enum with an
associated ErrorSpace, then the canonical space will be used.
You can also use `google3` permanent callback as an action. Note that `Callback`
or member function must be wrapped with `Invoke()`, whereas lambdas and functors
will work by themselves.
#### Using Callbacks as Matchers
Callbacks are widely used in `google3`. Conceptually, a `ResultCallback1<bool,
T>` is just a predicate on argument of type `T`. Naturally, we sometimes would
want to use such a callback as a matcher.
gMock gives you two function templates in namespace `testing` to turn callbacks
into matchers.
The first is `Truly(callback)`. It matches `argument` iff
`callback->Run(argument)` returns `true`.
The second is `AddressSatisfies(callback)`, which matches `argument` whenever
`callback->Run(&argument)` returns `true`.
The callbacks used in `Truly()` and `AddressSatisfies()` must be permanent (e.g.
those returned by `NewPermanentCallback()`), or you'll get a run-time error. The
matcher takes ownership of the callback, so you don't need to worry about
deleting it.
For examples, see
google3/testing/base/internal/gmock_utils/callback-matchers_test.cc.
#### Mock(able) Files {#MockableFile}
Don't use Mockable Files except to simulate errors on File. For general testing
of File, see go/file-testing.
google3/file/testing/mockablefile/mockablefile.h defines a `MockableFile` class.
It wraps an arbitrary `File` object and makes its virtual methods "mockable",
meaning that by default they'll delegate to the underlying `File` while you have
the option to call `ON_CALL` or `EXPECT_CALL` to set expectations on them and/or
change their behavior. This gives you the best part of both a mock and a real
object:
* The methods all have a working, default implementation. This can greatly
reduce the amount of work needed to set up the mock object.
* By setting expectations on the methods using `EXPECT_CALL`, you can easily
test how your code uses the `File`.
* By changing the methods' behavior (using `ON_CALL`), you can easily simulate
file errors and thus test your error handling code.
`mockablefile.h` contains copious comments on the usage, and
[`mockablefile_test.cc`](http://google3/file/testing/mockablefile/mockablefile_test.cc)
in the same directory contains some complete examples. Here's one of them,
showing how to simulate `Write()` errors:
```cpp
#include "file/base/path.h"
#include "file/testing/mockablefile/mockablefile.h"
using ::file::testing::MockableFile;
using ::testing::_;
using ::testing::DoDefault;
using ::testing::Return;
// This test demonstrates using MockableFile to test code that handles
// File operation errors. We want to test that WriteToFile() returns
// false when there is a Write() failure. It's hard to cause such an
// error using a real File object, but easy to make MockableFile
// simulate it.
TEST(SampleTest, SimulateFileError) {
// Creates a mockable_file object from a real File object. The real
// file is a local file in this example, but can also be any other
// type of File.
MockableFile* const mockable_file = new MockableFile(
File::Create(file::JoinPath(FLAGS_test_tmpdir, "/test"), "w"));
// Tells the mockable file to start failing from the second Write()
// operation on.
EXPECT_CALL(*mockable_file, Write)
// By default, calls are delegated to the real File object.
.WillOnce(DoDefault())
// Simulates a write error from the second time on.
.WillRepeatedly(Return(util::UnknownError("message")));
// Exercises the code we want to test, letting it talk to the
// MockableFile object instead of a real one.
EXPECT_FALSE(WriteToFile(mockable_file));
}
```
`mockablefile.h` also defines a `MockableFileSystem` class that allows you to
register mockable files in the file system under the `/mockable` mount point,
which can then be opened by your code by name. Since `MockableFile` can wrap
**any** type of file, this means you can inject **any** type of file into the
file system for testing. For example, `google3/file/memfile/memfile.h` defines a
convenient in-memory file type `MutableStringFile`. Now, you can wrap a
`MutableStringFile` in a `MockableFile` and inject it using `MockableFileSystem`
in order to test error handling of File operations that want to open a file
themselves.
```cpp
#include "file/memfile/memfile.h" // you also need to depend on //file/memfile:memfile in your BUILD file
#include "file/testing/mockablefile/mockablefile.h"
using ::file::testing::MockableFile;
using ::file::testing::MockableFileSystem;
using ::testing::_;
using ::testing::DoDefault;
using ::testing::Return;
// This test demonstrates registering a MockableFile with (a.k.a.
// injecting it into) the file system and opening it by name later.
// We want to test that WriteToFileByName() returns false when there
// is a Write() failure.
TEST(SampleTest, RegisterMockableFile) {
// Creates a mockable_file from a MutableStringFile.
MockableFile* const mockable_file = new MockableFile(
MutableStringFile("/foo/bar", new string,
TAKE_OWNERSHIP, DO_NOT_ALLOW_MMAP));
// Creates a mockable file system so that we can inject
// mockable_file into it.
MockableFileSystem fs;
// Injects mockable_file as "/mockable/foo/bar".
const string kPath = "/mockable/foo/bar";
EXPECT_CALL(fs, CreateFile(kPath, "w", _, _, _))
.WillOnce(Return(mockable_file));
// Tells the mockable file to start failing from the second Write()
// operation on.
EXPECT_CALL(*mockable_file, Write)
// By default, calls are delegated to the real File object.
.WillOnce(DoDefault())
// Simulates a write error from the second time on.
.WillRepeatedly(Return(util::error::UNKNOWN));
// Exercises the code we want to test, letting it talk to the
// MockableFile object instead of a real one.
EXPECT_FALSE(WriteToFileByName(kPath));
}
```
#### Mock Network System Calls
Gary Morain (gmorain@) implemented mock network system calls such that you can
use gMock to control their behavior when testing code that invokes network
system calls. You can find the code here:
* google3/net/util/network_system_call_interface.h - the interface.
* google3/net/util/network_system_call.h - the real implementation.
* google3/net/util/network_system_call_mock.h - the mock implementation.
* google3/net/util/network_system_call_unittest.cc - the unit test and demo.
#### Mock Bigtable
Please see the
[Svala](https://sites.google.com/a/google.com/te-zrh/tools--technologies/gmock-bigtable)
project for a gMock-based Bigtable implementation.
#### Add Yours Here
Don't be shy! If you've created a mock class using gMock and think it would be
useful to other Googlers, write an entry about it on this wiki page so that
people can learn about it.
#### Matching Protocol Buffers
Many Google APIs pass protocol buffers around. gMock provides some matchers for
protocol buffers. You can use them to specify that an argument must be equal (or
equivalent) to a given protocol buffer.
`EqualsProto(proto_buffer)` matches an argument iff it's equal to
`proto_buffer`, as determined by the `Equals()` method of the argument. The
argument must be a protocol buffer; pointers must be dereferenced.
Sometimes we want to test for equivalence instead of equality, i.e. we want to
use the `Equivalent()` method to compare two protocol buffers. For this we can
use `EquivToProto(proto_buffer)`.
It's worth noting that all of the matchers we mention here make a copy of
`proto_buffer`. This means that you can use a matcher even if the original
protocol buffer used for creating the matcher has been destroyed. Just one less
thing for you to worry about!
Note that `EqualsProto` and `EquivToProto` work for both proto1 and proto2. They
are declared in `gmock.h`, so you do not have to include other files. See
go/protomatchers for more proto buffer matching goodies.
In addition: One application of `Property()` is testing protocol buffers:
<a name="table1"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `Property(&MyProto::has_size, true)` </td>
<td> Matches `proto` where `proto.has_size()` returns `true`. </td>
</tr>
<tr>
<td> `Property(&MyProto::size, Gt(5))` </td>
<td> Matches `proto` where `proto.size()` is greater than 5. </td>
</tr>
</table>
### I need to mock a Stubby server. Should I use gMock or the service mocker? {#GMockVsServiceMocker}
To quote PiotrKaminski, the author of the service mocker:
You can find an introduction to the service mocker
[here](http://go/stubby-codelab#test-client), and detailed documentation in
net/rpc/testing/public/servicemocker.h. As I'm the author of the framework my
opinion on it can hardly be objective, but here are the main advantages it has
over gMock when it comes to mocking Stubby services:
* Services are mocked dynamically so there's no need to manually write mock
service implementations.
* The client's calls go through a real Stubby channel, which will catch some
errors that calling a service implementation directly would miss.
* The service mocker is aware of sync/async client distinctions and common
Stubby threading strategies, and in general allows you to exert more control
over when the callback is made.
* The base syntax and semantics are very similar to gMock, but Stubby-specific
matchers and actions make the testing code more compact.
* A powerful expectation grouping mechanism allows expressing complicated
async call ordering constraints in a readable fashion.
* By the end of the week, there'll be built-in support for testing call
cancellation.
Some disadvantages:
* The service mocker documentation is not as good as gMock's.
* The error messages are probably not as good as gMock's either.
* You can only mock services, not arbitrary classes. Expectations do not
interact with gMock's.
* Slightly different expectation matching semantics in corner cases, which
could get confusing if you're using gMock as well.
In my biased opinion, if you only need to mock out Stubby services, you should
look at the service mocker first. If you need to mock out other classes too, and
especially if you need to express relationships between service and other
expectations, you're probably better off with gMock.
#### Mock Callbacks
Callbacks (`"base/callback.h"`) are widely used in `google3` to package data and
logic together. Sometimes you want to test how your code invokes callbacks (with
what arguments, how many times, in which order, and etc). This is a job cut out
for mock callbacks.
`"testing/base/public/mock-callback.h"` defines a class template to mock
callbacks. Given arbitrary types `R`, `T1`, ..., and `Tn`, class
**`MockCallback<R(T1, ..., Tn)>`** mocks a callback that takes arguments of type
`T1`, ..., and `Tn`, and returns type `R`, which can be `void`. This class is
derived from its corresponding abstract callback classed defined in
`"base/callback.h"`, for example:
* `MockCallback<void()>` inherits from `Closure`,
* `MockCallback<void(int, double)>` inherits from `Callback2<int, double>`,
* `MockCallback<int()>` derives from `ResultCallback<int>`, and
* `MockCallback<string(bool)>` derives from `ResultCallback1<string, bool>`.
Compared with the various classes in `"base/callback.h"`, the mock classes share
the same name and only differ in the template arguments, so you will never have
trouble remembering which is called what.
Like a real callback, a mock callback can be either *single-use* or *permanent*.
A single-use mock callback will delete itself when invoked. A permanent mock
callback will not and thus can be invoked many times - you have to make sure it
is deleted somehow.
Since a mock object verifies all expectations on its mock methods in the
destructor, please link with `//base:heapcheck` (it is already linked
automatically if you link with `//testing/base/public:gtest_main`) to make sure
all mock callbacks
are properly deleted.
`MockCallback<R(T1, ..., Tn)>` has a mock method `OnRun()` with the signature:
```cpp
R OnRun(T1, ..., Tn);
```
`OnRun()` will be called whenever the mock callback is invoked. Note that we
don't name it `Run()` to match the method in the base class, as doing so will
interfere with mocking single-use callbacks.
Finally, `"mock-callback.h"` is a header-only library, so just include it and
go. Here's a complete example on how you use it:
```cpp
#include "testing/base/public/mock-callback.h"
// 1. Import the necessary names from the testing name space.
using ::testing::_;
using ::testing::MockCallback;
using ::testing::NotNull;
using ::testing::NewMockCallback;
using ::testing::NewPermanentMockCallback;
using ::testing::SetArgPointee;
TEST(FooTest, DoesBar) {
// 2. Create a single-use mock callback using NewMockCallback(), or
// a permanent mock callback using NewPermanentMockCallback().
MockCallback<string(int n, bool show_sign)>* show_int = NewMockCallback();
std::unique_ptr<MockCallback<void(int* count)> > get_count(
NewPermanentMockCallback());
// 3. Set expectations on the OnRun() method of the mock callbacks.
EXPECT_CALL(*show_int, OnRun(5, true))
.WillOnce(Return("+5"));
EXPECT_CALL(*get_count, OnRun(NotNull()))
.WillOnce(SetArgPointee<0>(1))
.WillOnce(SetArgPointee<0>(2));
// 4. Exercise code that uses the mock callbacks. The single-use
// mock callback show_int will be verified and deleted when it's
// called. Link with //base:heapcheck to make sure it is not
// leaked.
Foo(5, show_int, get_count.get());
// Foo()'s signature:
// void Foo(int n, ResultCallback2<string, int, bool>* show_int,
// Callback1<int*>* get_count);
// 5. The permanent mock callback will be verified and deleted here,
// thanks to the std::unique_ptr.
}
```
Did you notice that you don't specify the types when calling `NewMockCallback()`
and `NewPermanentMockCallback()`? Apparently they can read your mind and know
the type of the mock callback you want. :-)
(Seriously, these functions figure out their return types from the
left-hand-side of the assignment or the initialization, with the help of some
template tricks. But you don't need to understand how they work in order to use
mock callbacks.)
### Can I use gMock in multi-threaded programs?
googletest was designed with thread-safety in mind. It uses synchronization
primitives from `google3` to be thread-safe. If you work in `google3`, you can
use gMock in multiple threads safely. If you work outside of `google3` and need
gMock to be thread-safe, please let us know.
For more details on how to use gMock with threads, read this
[recipe](#UsingThreads).
### Protocol Buffer Matchers {#ProtoMatchers}
(go/protomatchers)
In the following, `argument` can be either a protocol buffer (version 1 or
version 2) or a pointer to it, and `proto` can be either a protocol buffer or a
human-readable ASCII string representing it (e.g. `"foo: 5"`). If you need help
writing the ASCII string, read go/textformat. "Fully-initialized" below means
all `required` fields are set.
<a name="table15"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `EqualsInitializedProto(proto)` </td>
<td> `argument` is fully-initialized and equal to `proto`. </td>
</tr>
<tr>
<td> `EqualsProto(proto)` </td>
<td> `argument` is equal to `proto`. Can also be used as a multi-argument matcher; see below. </td>
</tr>
<tr>
<td> `EquivToInitializedProto(proto)` </td>
<td> `argument` is fully-initialized and equivalent to `proto`. </td>
</tr>
<tr>
<td> `EquivToProto(proto)` </td>
<td> `argument` is equivalent to `proto`. Can also be used as a multi-argument matcher; see below. </td>
</tr>
<tr>
<td> `IsInitializedProto()` </td>
<td> `argument` is a fully-initialized protocol buffer. </td>
</tr>
</table>
Both Equiv and Equal matchers checks that two protocol buffers have identical
values, however only Equal matchers ensure that the protocol buffers fields were
set the same way (explicitly or through their default value).
When these matchers are given a string parameter, they *optionally* accept the
type of the protocol buffer as a template argument, e.g.
`EqualsProto<MyPB>("bar: 'xyz'")`.
The following *protocol buffer matcher transformers* in namespace
`::testing::proto` change the behavior of a matcher:
<a name="table16"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `Approximately(proto_matcher)` </td>
<td> The same as `proto_matcher` except that it compares floating-point fields approximately. </td>
</tr>
<tr>
<td> `Approximately(proto_matcher, margin)` </td>
<td> The same as `Approximately(proto_matcher)` except that two floating-point fields are considered equal if their absolute difference is <= `margin`. </td>
</tr>
<tr>
<td> `Approximately(proto_matcher, margin, fraction)` </td>
<td> The same as `Approximately(proto_matcher)` except that two floating-point fields are considered equal if their absolute difference is <= `margin` or their fractional difference is <= `fraction`. </td>
</tr>
<tr>
<td> `TreatingNaNsAsEqual(proto_matcher)` </td>
<td> The same as `proto_matcher` except that two floating-point fields are considered equal if both are NaN, matching the behavior of `NanSensitiveDoubleEq()`. </td>
</tr>
<tr>
<td> `IgnoringRepeatedFieldOrdering(proto_matcher)` </td>
<td> The same as `proto_matcher` except that it ignores ordering of elements within repeated fields (see `proto2::MessageDifferencer::TreatAsSet()` for more details). </td>
</tr>
<tr>
<td> `IgnoringFieldPaths({"some_field.subfield"}, proto_matcher)` </td>
<td> The same as `proto_matcher` except that it ignores the value of field `subfield` in field `some_field`. </td>
</tr>
<tr>
<td> `Partially(proto_matcher)` </td>
<td> The same as `proto_matcher` except that only fields present in the expected protocol buffer are considered. </td>
</tr>
<tr>
<td> `WhenDeserialized(typed_proto_matcher)` </td>
<td> `argument` is a string in the protocol buffer binary format that can be deserialized to a protocol buffer matching `typed_proto_matcher`. </td>
</tr>
<tr>
<td> `WhenDeserializedAs<PB>(matcher)` </td>
<td> `argument` is a string in the protocol buffer binary format that can be deserialized to a protocol buffer of type `PB` that matches `matcher`. </td>
</tr>
<tr>
<td> `WhenParsedFromProtoText(typed_proto_matcher)` </td>
<td> `argument` is a string in the protocol buffer text format that can be parsed to a protocol buffer matching `typed_proto_matcher`. </td>
</tr>
<tr>
<td> `WhenParsedFromProtoTextAs<PB>(matcher)` </td>
<td> `argument` is a string in the protocol buffer text format that can be parsed to a protocol buffer of type `PB` that matches `matcher`. </td>
</tr>
<tr>
<td> `WhenUnpacked(typed_proto_matcher)` </td>
<td> `argument` is a `google.protobuf.Any` that can be unpacked into a protocol buffer of the type of `typed_proto_matcher` that matches that matcher. </td>
</tr>
<tr>
<td> `WhenUnpackedTo<PB>(matcher)` </td>
<td> `argument` is a `google.protobuf.Any` that can be unpacked into a protocol buffer of type `PB` that matches `matcher`. </td>
</tr>
</table>
where:
* `proto_matcher` can be any of the `Equals*` and `EquivTo*` protocol buffer
matchers above,
* `typed_proto_matcher` can be an `Equals*` or `EquivTo*` protocol buffer
matcher where the type of the expected protocol buffer is known at run time
(e.g. `EqualsProto(expected_pb)` or `EqualsProto<MyPB>("bar: 'xyz'")`).
* `matcher` can be any matcher that can be used to match a `PB` value, e.g.
`EqualsProto("bar: 'xyz'")`, `Not(EqualsProto(my_pb))`.
`Approximately()`, `Partially()`, and `TreatingNaNsAsEqual()` can be combined,
e.g. `Partially(Approximately(EqualsProto(foo)))`.
Note that `EqualsProto()` and `EquivToProto()` can be used as multi-argument
matchers that match a 2-tuple of protos. The following example shows how to
compare two vectors of protos.
```cpp
vector<MyProto> actual;
vector<MyProto> expected;
... // Fill vectors with data
EXPECT_THAT(actual, Pointwise(EqualsProto(), expected));
```
Similarly, they can be used to compare a vector of protos against a vector of
strings.
```cpp
vector<MyProto> actual;
... // Fill 'actual' with data
vector<string> expected {"foo:<bar:1>", "foo:<bar:2>"};
EXPECT_THAT(actual, Pointwise(EqualsProto(), expected));
// Or, concisely:
EXPECT_THAT(actual, Pointwise(EqualsProto(), {"foo:<bar:1>", "foo:<bar:2>"}));
```
<a name="table17"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `EqualsProto()` </td>
<td> `x.Equals(y)` </td>
</tr>
<tr>
<td> `EquivToProto()` </td>
<td> `x.Equivalent(y)` </td>
</tr>
</table>
### Stubby Actions
gMock has the following actions to provide limited support for mocking Stubby
(go/stubby) services. You can use them to return a canned answer from a Stubby
call, which has the signature `void Method(RPC*, const Request*, Response*
response, Closure* done)`. You should consider using Service Mocker
(go/servicemocker) instead if your need is more complex.
<a name="table35"></a>
<table border="1" cellspacing="0" cellpadding="1">
<tr>
<td> `BeDone()` </td>
<td> Calls the `done` closure. </td>
</tr>
<tr>
<td> `FailWith(status)` </td>
<td> Fails the RPC with the given RPC status code. </td>
</tr>
<tr>
<td> `FailWithUtilStatus(util_status)` </td>
<td> Fails the RPC with the given util::Status error code. </td>
</tr>
<tr>
<td> `RespondWith(proto)` </td>
<td> Sets the `response` argument to the given protocol buffer, and calls the `done` closure. </td>
</tr>
<tr>
<td> `RespondWith(proto_string)` </td>
<td> Sets the `response` argument to the protocol buffer parsed from the given ASCII string, and calls the `done` closure. </td>
</tr>
</table>
#### Testing LOG()s {#TestingLogs}
LOG()s are widely used in `google3` programs. They make it possible to diagnose
a server crash when you don't have the luxury of reproducing the bug. They are
also great as a [tool for refactoring](http://go/log-pin).
Often we need to test how a piece of code calls LOG()s. Traditionally, this has
been done using [golden files](http://go/log-pin), which is tedious to set up
and brittle (what if a library you depend on starts to generate its own logs?).
The [`ScopedMemoryLog`](http://go/gunit-faq-scoped-mock-log) class was created
to allow writing robust LOG tests, but using it beyond the most basic scenario
can be awkward.
With gMock we have a better solution. `testing/base/public/mock-log.h` defines a
mock log sink class `ScopedMockLog`. A `ScopedMockLog` object intercepts all
LOG()s (except `LOG(FATAL)`) while it is alive. This object has a mock method of
this signature:
```cpp
void Log(LogSeverity severity, const string& path, const string& message);
```
This file comes with gUnit and gMock, so there is no need to add any dependency
to your `BUILD` rule in order to use it.
Here are some ideas on how to make use of it:
To test that the code generates exactly one warning message (and nothing else):
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log(WARNING, _, "Expected warning."));
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that a particular message is logged exactly once (but there can be other
log messages with different contents):
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::AnyNumber;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log).Times(AnyNumber());
EXPECT_CALL(log, Log(INFO, _, "Expected message"));
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that no `ERROR` is logged (but there can be other log messages with
different severities):
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::AnyNumber;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log).Times(AnyNumber());
EXPECT_CALL(log, Log(ERROR, _, _))
.Times(0);
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that a particular message is logged at least once (and there can be
other log messages):
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::AnyNumber;
using ::testing::AtLeast;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log).Times(AnyNumber());
EXPECT_CALL(log, Log(INFO, _, "Expected message"))
.Times(AtLeast(1));
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that three LOG()s occur sequentially:
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::InSequence;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
{
InSequence s;
EXPECT_CALL(log, Log(INFO, _, "Log #1"));
EXPECT_CALL(log, Log(WARNING, _, "Log #2"));
EXPECT_CALL(log, Log(INFO, _, "Log #3"));
}
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that the log message contains a certain sub-string:
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::HasSubstr;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log(WARNING, _, HasSubstr("needle")));
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that a given module generates a specific log:
```cpp
using ::testing::kDoNotCaptureLogsYet;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log(WARNING, "path/to/my_module.cc", "Expected warning."));
log.StartCapturingLogs();
... code that LOG()s ...
```
To test that code doesn't log anything at all:
```cpp
using ::testing::_;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::ScopedMockLog;
...
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log).Times(0);
log.StartCapturingLogs();
... code that does not LOG() ...
```
**Warning:** For robust tests, either ignore unexpected logs (loose), or ignore
logs in other modules (tight), otherwise your test may break if their logging
changes.
```cpp
using ::testing::_;
using ::testing::AnyNumber;
using ::testing::kDoNotCaptureLogsYet;
using ::testing::Not;
using ::testing::ScopedMockLog;
// ...
// Simple robust setup, ignores unexpected logs.
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log).Times(AnyNumber()); // Ignore unexpected logs.
EXPECT_CALL(log, Log(ERROR, "path/to/my_file.cc", _))
.Times(3); // Verifies logs from my_file.cc.
log.StartCapturingLogs();
// ... code that LOG()s ...
// ...
// Tighter alternative.
ScopedMockLog log(kDoNotCaptureLogsYet);
EXPECT_CALL(log, Log(_, Not("path/to/my_file.cc"), _))
.Times(AnyNumber()); // Ignores other modules.
EXPECT_CALL(log, Log(ERROR, "path/to/my_file.cc", _))
.Times(3); // Verifies logs from my_file.cc.
log.StartCapturingLogs();
// ... code that LOG()s ...
```
To test `LOG(DFATAL)`, use
[`EXPECT_DFATAL`](/third_party/googletest/googletest/docs/google3_faq#testing-death-in-debug-mode)
instead.
#### Testing Code that Uses a Stubby Server
(Contributed by JonWray on 2008/3/11; updated by ZhanyongWan later.)
I'm testing a C++ frontend that calls several different backends, but I'll just
include an example for one to keep this relatively short. This example is
mocking a `CacheServer` backend. An explanation follows the code.
```cpp
using ::testing::_;
using ::testing::BeDone;
using ::testing::EqualsProto;
using ::testing::RespondWith;
class MockCacheServer : public CacheServerRPC {
public:
MockCacheServer(HTTPServer *hs) {
rpc2::EnableRPC2(this, rpc2::ServiceParameters());
CacheServerRPC::ExportService(hs);
ON_CALL(*this, Insert).WillByDefault(BeDone());
ON_CALL(*this, Lookup).WillByDefault(BeDone());
}
MOCK_METHOD(void, Insert,
(RPC*, const CacheInsertCommandProto*, CacheInsertResultsProto*,
Closure*),
(override));
MOCK_METHOD(void, Lookup,
(RPC*, const CacheLookupCommandProto*, CacheLookupResultsProto*,
Closure*),
(override));
};
...
// This is in the test fixture.
MockCacheServer cacheserver_;
...
// Now the code that uses it:
CacheLookupCommandProto command;
// Init command
CacheLookupResultsProto results;
// Init results
EXPECT_CALL(cacheserver_, Lookup(_, EqualsProto(command), _, _))
.WillOnce(RespondWith(results));
```
In the success case, the command matches the `EXPECT_CALL`, so results is set
and the callback is called.
In the failure case, the command matches the default `ON_CALL`, the results are
not set, and the done closure is called (don't want the test to hang).
So it's a bit ugly, but given that I need to mock five backends, I think it's
better than doing this manually. The best part is the nicely formatted error
messages when the expected call is incorrect. Once all this scaffolding is in
place, it's easy to churn out test suites.
**Discussions:**
* ZhanyongWan: `StubbySimulator` by Mike Bland might also be useful:
google3/testing/lib/net/rpc/stubby_simulator.h.
* JonWray: This is turning a mock into a fake, isn't it? All requests are
accepted, and you can write logic to set the reply conditionally on the
request. The interesting thing is the logic moves from the mock class into
the test suite.
* MikeBland: It's sort of a Frankenstein, but it works well for my purposes.
It collaborates with a mock Stubby server, which sets the expectation and
does the actual intercepting of the args, and then gives you the freedom to
fiddle with the results while the RPC is conceptually "in flight". This is
especially handy for "pausing" the RPC and having it return a state other
than `RPC::OK`. This sort of approach to splitting RPC calls into separate
objects from ControlFlows was first explored in
[TotT Episode 46](http://tott/2007/06/episode-46-encapsulated-rpcs-or.html).
* PiotrKaminski: The [Service Mocker](http://go/servicemocker) is a gMock-like
framework that specializes in mocking Stubby services. It allows for small,
in-process tests, doesn't require manually writing a service mock, and can
deal with async clients, streaming and testing for cancellations.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment