# Environmental Estimation With Distributed Finite Element Agents

Matthew L. ElwinRandy A. FreemanKevin M. Lynch

Abstract— We develop an environmental estimation method that allows large groups of agents to infer the value of an environmental ﬁeld using measurements. Agents maintain estimates for subregions of the domain and communicate with local neighbors, so the method’s communication and memory requirements do not increase with the number of agents, the size of the environment representation, or the agents’ density in the environment. Despite the distributed representation, the union of individual estimates matches an estimate generated by a central computer with access to all measurements employing the variational inverse method, a ﬁnite element-based interpolation procedure. We also introduce a distributed query system, allowing users to determine an estimate anywhere in the domain without accessing all measurements or the full environment representation.

I. INTRODUCTION A common problem in sciences such as oceanography, earth science, and atmospheric science is to infer the value of a scalar ﬁeld (e.g., temperature, chemical concentration, radiation intensity) anywhere in an environment based on irregularly spaced measurements. Typically, the relatively sparse data are processed by a central computer; such meth- ods have been studied extensively (see e.g., [1]). The advent of cheap sensors, however, presents an opportunity to collect more data than ever before. As the number of sensors used to measure an environment increases, so do computation and communication burdens. These problems become especially acute in remote environments such as the ocean, the polar icecaps, outer space, and underground. In such environments, communication with a distant base station may be signiﬁ- cantly harder than communication between agents, making it impractical to send all measurements to a central computer. To address the problem of handling many measurements with limited memory and bandwidth, we develop a dis- tributed version of the variational inverse method (VIM) for data interpolation [2]–[6]. This technique, used in oceanog- raphy, generates an estimate by minimizing a cost functional using the Finite Element Method (FEM) [7]. Rather than having a central computer collect data from sensors, we deploy the sensors on agents that communicate with each other and perform computations. Each agent determines an estimate for a small portion of the environment. No single agent, therefore, stores a full environmental representation



This work is supported by the Ofﬁce of Naval Research, Grant N00014-13-1-0331. The authors are with the Department of Mechanical Engineering (Elwin and Lynch), the Northwestern Institute on Complex Systems (Freeman and Lynch), and the Department of Electrical Engineering and Computer Science (Freeman), Northwestern University, Evanston, IL60208USA.Emails:elwin@u.northwestern.edu, freeman@eecs.northwestern.edu, kmlynch@northwestern.edu.

or all of the measurements. By communicating with local neighbors, however, the collection of all agents’ estimates matches the result obtained by a central computer using VIM. We use the alternating direction method of multipliers (ADMM) to distribute computations and solve the FEM problem [8]. Other methods, which generally require more communication than ADMM include [9] and [10].

We couple our distributed VIM (DVIM) method with a query system, allowing a user to determine the value of the ﬁeld anywhere. The user can obtain a low-resolution view of the overall ﬁeld and then focus on higher resolution estimates in areas of particular interest. Using this system, the full set of measurements and a complete environment representation are never communicated to or stored by a single computer. When communication with a remote base station (e.g., via satellite) costs signiﬁcantly more than inter- agent communication (e.g., via XBee), minimizing such communication becomes especially important.

In Section II we compare DVIM to existing methods in the literature. In Section III we formulate the problem, intro- ducing VIM and FEM. Section IV describes our distributed solution, DVIM, and the query system. Section  $\mathrm{V}$  provides simulation results. We outline our conclusions and future work in Section VI.

II. METHOD COMPARISON We compare DVIM to existing techniques whose com- putation, communication, and memory requirements do not increase with respect to the number of agents. Many criteria can be used to evaluate methods, but in this work we are particularly concerned with three desirable properties: scalability with respect to the environment representation, scalability with respect to agent density, and incorporation of all relevant measurements.



Scalability with respect to the environment representa- tion means that every agent’s memory and communication requirements do not increase with the resolution of the environment model. Methods that scale with the environment representation allow adding more agents with ﬁxed capabil- ities to obtain higher resolution estimates.

Scalability with respect to agent density means that every agent’s memory and communication requirements do not increase as the density of the agents in the environment (and therefore measurement density) increases. High measurement density contributes to more accurate estimates, so methods with this property provide opportunities for improved accu- racy by adding more agents with ﬁxed capabilities.

Methods that incorporate all relevant measurements gen- erate an estimate at a given point using measurements from

ENVIRONMENT REPRESENTATION, SAD - SCALE WITH AGENT DENSITY, IRM - INCORPORATE RELEVANT MEASUREMENTS.

locations correlated with that point. Many methods assume that the correlation between points decreases with distance, and that points further than a characteristic correlation length apart are uncorrelated. Although VIM (and therefore DVIM) does not rely on this assumption (an advantage; it can handle cases when nearby points should be uncorrelated, see [3]), it incorporates all measurements when forming estimates and does have a notion of spatial correlation [4]. Methods that disregard measurements that should inﬂuence an estimate at a given point do not use information efﬁciently.

Two features of all the methods we examine that DVIM currently lacks are control laws for mobile agents and error estimation abilities (we leave these for future work). To ensure a fair comparison, we assume, for all papers, station- ary agents and known error ﬁelds for all agents. Although estimating an error ﬁeld provides beneﬁts, it also imposes additional communication and memory requirements on the agents. Additionally, in [11], movement increases communi- cation and memory requirements because the environmental representation is coupled to the agents’ trajectories.

We now describe several methods and evaluate whether they scale with the environment representation, scale with agent density, and incorporate all relevant measurements. Table I shows this comparison, with more details provided below. Note that a central computer collecting measurements from all agents and applying VIM does not scale with the environment representation or with agent density.

A. Distributed Kalman Filter [12] This method uses a distributed Kalman ﬁlter based on average consensus estimators to incorporate measurements and estimate basis function coefﬁcients that represent the en- vironment. The method could also work using the distributed data fusion techniques of [16]. Communication and memory is proportional to the number of basis functions, which is ﬁxed beforehand; therefore, this method does not scale with the environment representation. Every agent, however, has a full environment estimate. This method scales with agent density because it works over any connected communication graph. The relationship between measurements at different points is incorporated in the basis function choice, so mea- surements contribute to the ﬁeld estimate everywhere.



B. Kriged Kalman Filter [13] This method represents the environment as the mean and covariance functions of a spatial Gaussian process. The representation consists of basis function coefﬁcients (as in [12]) and as a combination of measurements within a given radius  $R$  of each agent. If the coefﬁcients are unknown, communication and memory is proportional to the number of basis functions; however, by assuming known coefﬁcients, communication is independent of the global environment description and this method can scale with the environment representation. Regardless, the communication increases with density: estimation requires the agents to incorporate all measurements within a radius of  $R$ . Since the agents account for measurements within a radius of  $R$  and points further than  $R$  apart are assumed to be uncorrelated, all relevant measurements contribute to the estimate.

 



## C. Non-parametric Information  $I I4J$ 

This method represents the environment as a weighted set of discrete samples from a probability distribution. Accord- ingly, it can approximate any type of environment, not just scalar ﬁelds. Every agent, however, has a complete envi- ronment representation and must communicate information proportional to the size of this representation. This method, therefore, does not scale with the representation size. It does, however, scale with agent density because it works over connected graphs. Information is fused using a particle-ﬁlter like approach, so the estimate at a given location incorporates relevant measurements.

## D. Local Interpolation: Nearest Neighbor [11]

In this method, the environment is a piecewise constant function. When used with stationary agents, an agent’s estimate over its Voronoi region is its measurement. This method, therefore, scales with the environment representation (adding more agents adds more Voronoi cells) and with density. The nearest neighbor approach, however, uses a single measurement to estimate the ﬁeld over each Voronoi cell, so it does not use all relevant measurements. With mobile agents, as originally intended, the initial Voronoi cells are subdivided based on current and past measurements, creating a more detailed estimate. The environment represen- tation, however, is coupled to the agent’s trajectories; lossy compression is used to maintain scalability with respect to the number of measurements when the agents move.

E. Local Interpolation: Inverse Distance [11] Closely related to Nearest Neighbor Local Interpolation, this method uses piecewise continuous functions to represent the environment. Here, measurements have a diminishing effect on the estimate as distance from the measurement de- creases. Thus, unlike the Nearest Neighbor version, estimates at a given point incorporate relevant measurements. However, now the estimate must incorporate all measurements from within a given radius  $R$ , so the method no longer scales with agent density.



##  $F.$  Distributed Finite Element Kalman Filter [15], [17]

In this method, the environment is modeled as a partial differential equation (PDE) in space and time. Using ﬁnite elements over the spatial variables and discretizing over time, the PDE is converted into a discrete time state-space system and a Kalman Filter used to estimate the state. Each agent is responsible for only a subset of the domain; thus the method scales with respect to the environmental model. At each time-step, the state is estimated using the parallel Schwarz method and measurements are incorporated using a Kalman ﬁlter update. The method requires only information shared by adjacent regions to be exchanged and converges to a centralized solution; therefore the method scales with agent density and incorporates all relevant information.



G. Distributed Variational Inverse Method This method, as developed here, is scalable with respect to environment representation, agent density, and incorpo- rates relevant measurements when forming an estimate. The environment representation is the total number of nodes in a global FEM mesh, but each agent handles only a subset of those nodes. As the density of the agents increases, communication costs do not increase because communication is proportional to the number of Voronoi neighbors and the perimeter of every agent’s Voronoi region, which do not increase with density. Finally, in each agents’ region, the estimate matches the centralized VIM method, which incorporates all measurements into the estimate.



Unlike [15], [17], our method does not require a model of the ﬁeld; thus it is applicable in situations when partial differential equations for the environment are not known. Additionally, we present a method to form consistent ﬁnite element meshes in a distributed manner and a query system to extract the estimate, issues that are not addressed in [15], [17].

III. VARIATIONAL INVERSE METHOD

## A. Sensor and Environment Model

The environment is a scalar ﬁeld  $\bar{\phi}(x,y)$  over a region . Stationary agents dispersed over $\Omega\subset\mathbb{R}^{2}$   $\Omega$ , with agent  $i$  located at  $(x_{i},y_{i})\in\Omega$ , inaccurately measure the ﬁeld: of the estimate from measurements. In this paper we use




$$
\begin{array}{c}{{S[\phi]=\displaystyle\left(\frac{\partial^{2}\phi}{\partial x^{2}}\right)^{2}+\displaystyle\left(\frac{\partial^{2}\phi}{\partial y^{2}}\right)^{2}+2\displaystyle\left(\frac{\partial^{2}\phi}{\partial x\partial y}\right)^{2}}}\\ {{\displaystyle+\alpha_{1}\left(\left(\frac{\partial\phi}{\partial x}\right)^{2}+\left(\frac{\partial\phi}{\partial y}\right)^{2}\right)+\alpha_{0}(\phi-\phi_{0})^{2}}}\end{array}
$$


and


$$
D[\phi]=\sum_{k=1}^{n}\mu_{k}\left(\phi\big(x_{k},y_{k}\big)-z_{k}\right)^{2}
$$


(4) where  $\phi_{0}(x,y)$  is a known background ﬁeld,  $\alpha_{1}$  is a weight on the curvature,  $\alpha_{0}$  weights the error of the ﬁeld, and  $\mu_{k}$  weights each measurement.



The known background ﬁeld  $\phi_{0}(x,y)$  is the assumed value of the ﬁeld in the absence of measurements; it allows us to incorporate prior assumptions about the ﬁeld into the estimate. Alternatively,  $\phi_{0}(x,y)$  can be taken to be the average of the data, a linear ﬁt to the data, or zero. When  it makes sense to let $\phi_{0}(x,y)=0$   $\alpha_{0}=0$  to avoid penalizing the magnitude of the resulting estimated ﬁeld.

 $D.$ . Finite Element Method We use FEM to minimize the cost functional (2); details can be found in [7]. FEM consists of the following ﬁve steps: 1) Divide the domain into sub-domains called elements. 2) Find interpolating equations for each element based on values at points within each element, called nodes. 3) Assemble elements into a global system of equations. 4) Solve the equations. 5) Post-process the result to obtain information from the solution, for example, producing a visualization. 1) Dividing the domain: The domain is partitioned into a mesh of  $m$  non-overlapping sub-domains called elements (see Figure 2). Each element’s domain is  $\Omega_{e}$  and the elements approximately cover  $\Omega$ . Elements only intersect at vertices or along full edges. Many element geometries are available. For simplicity, we use a mesh of triangular elements, as gen- erated by the Computational Geometry Algorithms Library (CGAL) [18].

2) Interpolating equations: The estimate within each ele- ment,  $\phi_{e}(x,y)$ , is approximated by a polynomial containing all terms up to order  $d$ , which can be fully described by  $r$  coefﬁcients. This polynomial’s coefﬁcients are related to the value of  $\phi_{e}(x,y)$  at points within the element called nodes:  $\mathbb{R}^{r}$ ere  $w(x,y)$  is the vector of  $r$  shape functions and  $q_{e}\,\in$  Rr is the vector of nodal values. Many shape functions are available. We use quadratic polynomials over triangular domains, so  $r\ \ =\ \ 2$  (see Figure 2). The polynomial is , where $b_{0}\!+\!b_{1}x\!+\!b_{2}y\!+\!b_{3}x^{2}\!+\!b_{4}x y\!+\!b_{5}y^{2}$   $b_{i}$  is a coefﬁcient. There are six coefﬁcients and therefore six nodes (one at each corner and one bisecting each edge).



Each shape function (an element of  $w(x,y))$  corresponds to a node: it is one at that node and zero at all other nodes. For example, with  $r=2$ , evaluating  $w^{T}(x,y)$  at element  $e$ ’s second node yields  $\left[0\quad1\quad0\quad0\quad0\quad0\right]$  1000 0 , which selects the second element of  $q_{e}$  in equation (5).

3) Assembly: Each nodal vector  $q_{e}$  is local to element  $e$ ; however, nodes on adjacent element edges overlap. Gener- ally, therefore, there are  $p<r m$  nodes in the ﬁnite element mesh. The global node vector  $q\ \in\ \mathbb{R}^{p}$ is related to the local node vectors by the Boolean stochastic gather matrices   $L_{e}\in\mathbb{R}^{r\times p}$ such that:


$$
q_{e}=L_{e}q.
$$


(6) By deﬁning when $\phi_{e}(x,y)=0$   $(x,y)\not\in\Omega_{e}$ , an approxi- mation of the whole ﬁeld is




$$
\phi(x,y)\approx\sum_{i=1}^{m}\phi_{e}(x,y).
$$


(7) Note that by using the Dirac delta  $\delta$ , all terms of the cost functional (2) can be written inside the integral as




$$
J[\phi]=\int_{\Omega}\left(S[\phi]+D^{\prime}[\phi]\right)d\Omega,
$$


(8) where




$$
D^{\prime}[\phi]=\sum_{k=1}^{n}\mu_{k}\bigl(\phi-z_{k}\bigr)^{2}\delta\bigl(x-x_{k}\bigr)\delta\bigl(y-y_{k}\bigr).
$$


Substituting equation (7) into the cost functional (9) and performing some manipulations yields


$$
J[\phi]\approx\sum_{e=1}^{m}(q_{e}^{T}K_{e}q_{e}-2q_{e}^{T}g_{e})+\sum_{k=1}^{n}\mu_{k}(z_{k}-\phi_{0}(x_{k},y_{k}))^{2},\eqno(42)
$$


with


$$
\begin{array}{c}{{\displaystyle K_{c}=\int_{\Omega_{c}}\left[\left(\frac{\partial^{2}w}{\partial x^{2}}\right)\left(\frac{\partial^{2}w}{\partial x^{2}}\right)^{T}+\left(\frac{\partial^{2}w}{\partial y^{2}}\right)\left(\frac{\partial^{2}w}{\partial y^{2}}\right)^{T}\right.}}\\ {{\displaystyle\left.+\,2\left(\frac{\partial^{2}w}{\partial x^{2}\partial y}\right)\left(\frac{\partial^{2}w}{\partial x\partial y}\right)^{T}+\alpha_{1}\left(\frac{ 
$$


and


$$
g_{e}=\sum_{(x_{k},y_{k})\in\Omega_{e}}\mu_{k}w(x_{k},y_{k})z_{k}.
$$


(12) Substituting equation (6) into equation (10) and minimiz- ing with respect to  $q$  yields the global FEM equation




$$
K q=g,
$$


(13) where




$$
K=\sum_{e=1}^{m}L_{e}^{T}K_{e}L_{e}
$$


(14) and




$$
g=\sum_{e=1}^{m}L_{e}^{T}g_{e}.
$$


(15) 4) Solving: Solving equation (13) for  $q$  provides the ﬁeld estimate. For brevity we have omitted boundary conditions; implicitly assuming that derivatives of the estimate across the boundary are zero. However, our technique extends to cases when either the derivatives on the boundary or the value of some boundary nodes are speciﬁed.



5) Post-processing: Given the global node vector  $q$ , the ﬁeld can be estimated at any location using equations (7) and (6), allowing us to, for example, plot the estimate.

IV. DISTRIBUTED METHOD Our method, distributed VIM (DVIM), enables agents to estimate ﬁelds using local communication without any agent needing all measurements or a full environment representa- tion. The scaling properties of the algorithm, derived from its distributed nature, allow for more detailed environment rep- resentations as the group size increases without a concomi- tant increase in the memory or communication requirements of individuals. Memory and communication requirements also do not increase with increasing agent density. Individual agents’ estimates are consistent with VIM solved on a central computer with full access to all measurements.



To retrieve estimates from the group requires a query system. When at least one agent can communicate with a base station, users can query ﬁeld estimates at any location. Users may request estimates over a low-resolution grid and then focus on speciﬁc areas, allowing for less communication between the group and the base station.

Agents must perform the following four tasks: 1) Determine a region of dominance (ROD), the area over which they generate estimates. In this paper an agent’s ROD is the agent’s the Voronoi region. 2) Form a mesh within their (ROD) such that the union of all agents’ RODs forms a valid FEM mesh. We modify an existing meshing algorithm for this step. 3) Solve the FEM problem. We use the alternating direc- tion method of multipliers (ADMM) algorithm. 4) Respond to requests directed at determining the es- timate somewhere within the domain. We develop a direction-based depth ﬁrst search query system. These tasks can be repeated indeﬁnitely to provide estimates of time varying ﬁelds.

Determining the ROD (task 1) and forming the mesh (task 2) roughly correspond to the global FEM steps of dividing the domain (step 1) and ﬁnding the interpolating function (step 2). Task 3, where the agents solve the FEM problem, covers both the assembling (step 3) and solving (step 4) steps of global FEM. One advantage of using ADMM over other solving methods is that assembly happens locally on the agents; individual entries in  $K$  and  $g$  are never explicitly constructed by any agent. Task 4, querying, is the distributed version of post-processing (step 5).



We now discuss each step of distributed VIM individually. For clarity, we assume that every agent takes one measure- ment with one sensor, so there are  $n$  agents,  $n$  measurements, and  $n$  sensors; however, this method easily allows agents to take multiple measurements with multiple sensors.

 $I$ ) Region of Dominance: Each agent establishes a region of dominance (ROD)  $\Omega_{i}\,\subset\,\Omega$ , partitioning i $\Omega$ o  $n$  non- overlapping regions that approximately cover Ω. If two agents have intersecting RODs, they are neighbors and must be able to communicate.

WeuseboundedVoronoiregionsforestablishing RODs, deﬁned as the set of all points  $(x,y)$  such that  and $\|(x,y)-(x_{i},y_{i})\|<\|(x,y)-(x_{j},y_{j})\|$   $(x,y)\in\Omega$  for all  $i\neq j$ . Two agents are Voronoi neighbors if their Voronoi regions share an edge. If the agents know their absolute position and can communicate with their Voronoi neighbors, the algorithm of [19] allows distributed computation of the region. As argued in [19], this communication requirement results in scalable networks because, on average, each agent has, at most, six neighbors.

2) Consistent Mesh: Once the agents know their ROD, they must form a mesh consistent with their neighbors’ meshes. Although agents compute their meshes indepen- dently, agents with shared edges must place any mesh vertices on those edges at the same location. Assuming a polygonal ROD, every agent ﬁrst divides its edges into the minimum number of segments such that no segment exceeds some pre-determined maximum length  $\ell_{\mathrm{max}}$ . The vertices subdividing these edges will be vertices in the resulting mesh, and, due to symmetry, they will be at the same absolute location for adjacent agents.

Next, each agent runs the meshing algorithm of [18], which generates a mesh with no triangle edge lengths ex- ceeding  $\ell_{\mathrm{max}}$ . If the algorithm adds vertices on the ROD edges, they must be removed. Vertex removal is well-deﬁned on the structure returned by the meshing algorithm.

3) Distributed FEM: After determining its ROD and meshing it, every agent  $i$  has  $m_{i}$  elements and  $p_{i}$  nodes. Let  denote the set of elements contained in agent $E_{i}$   $i$ ’s ROD. Since elements are not shared between agents,  $E_{i}$  and  $E_{j}$  are disjoint for  $i\neq j$ . Thus, each agent has a regional node vector  $\bar{q}_{i}\;\in\;\mathbb{R}^{p_{i}}$  (analogous to  $q.$ ), which is related to the node vectors for each element in its region by


$$
q_{f}=\bar{L}_{f}\bar{q}_{i},\,\mathrm{for\all\}f\in E_{i},
$$


(16) where the stochastic Boolean matrix  $\bar{L}_{f}$  is analogous to  $L_{e}$ . Likewise, we form the regional matrices and $\bar{K}_{i}$   $\bar{g}_{i}$  anal- ogously to equations (14) and (15) according to




$$
\bar{K}_{i}=\sum_{f\in E_{i}}\bar{L}_{f}^{T}K_{f}\bar{L}_{f}
$$


(17) and




$$
\bar{g}_{i}=\sum_{f\in E_{i}}\bar{L}_{f}^{T}g_{f}.
$$


(18) The map between agent  $i$ ’s node vector and the global node vector is given by  $g(i,j)$  such that  $[\bar{q}_{i}]_{j}$  corresponds to , where $[q]_{g(i,j)}$   $[\cdot]_{j}$  indicates the  $j$ -th entry in a vector.



Using equations (10), (6), (17), and (18), we derive the following constrained minimization problem:


$$
\mathrm{argmin}\;\;\;\sum_{i=1}^{n}\left(\bar{q}_{i}^{T}\bar{K}_{i}\bar{q}_{i}-2\bar{q}_{i}^{T}\bar{g}_{i}\right)
$$


## with respect to  $\bar{q}_{i}$ , for  $i=1$  to n

subject to  $[\bar{q}_{i}]_{j}=[q]_{g(i,j)}$ , for  $i=1\,{\mathrm{to}}\,n$  and  $j=1$  to  $p_{i}$ . (19)

Note that the measurements enter the minimization prob- lem (19) via the term  $\bar{g}_{i}$ . The distributed ADMM method of [8] allows every agent to solve this problem for its own  $\bar{q}_{i}$ . Due to the constraints on  $\bar{q}_{i}$ , the union of all agents solutions is equivalent to solving the global FEM problem on the union of all agents’ meshes.

Let $N_{i,j}$ bethesetofallpairs $(k,l)$ suchthat . In other words, $[\bar{q}_{k}]_{l}=[q]_{g(i,j)}$   $N_{i,j}$  is the set of all agent- index, local-node-index pairs corresponding to the global node index. The cardinality of $g(i,j)$   $N_{i,j}$ ,  $|N_{i,j}|$  is then the number of local nodes corresponding to a given global node. To implement ADMM, every agent  $i$  iterates over three steps. We show a version speciﬁc to our problem derived from [8]:


$$
\begin{array}{l}{{[\xi_{i}(t)]_{j}=\displaystyle\frac{1}{|N_{i,j}|}\sum_{(k,l)\in N_{i,j}}[\hat{q}_{k}(t)]_{l},}}\\ {{\displaystyle\hat{q}_{i}(t+1)=(\bar{K}_{i}+\rho I)^{-1}(\bar{g}_{i}-\nu_{i}(t)+\rho\xi_{i}(t)),}}\\ {{\displaystyle\nu_{i}(t+1)=\nu_{i}(t)+\rho\left(\hat{q}_{i}(t)-\xi_{i}(t)\right),}}\end{array}
$$


where  $\rho>0$  is an optimization parameter,  $t$  is the discrete- time index,, $\xi_{i}(t)~\in~\mathbb{R}^{p_{i}}$   $\hat{q}_{i}(t)\ \in\ \mathbb{R}^{p_{i}}$ and  $\nu_{i}(k)~\in~\mathbb{R}^{p_{i}}$  are the state variables of the ADMM algorithm. The agents execute this algorithm repeatedly until a stopping condition is met; we use a ﬁxed number of iterations, but adopting the stopping criteria of [8] is left for future work. When the algorithm completes, the agents take new measurements and the process begins again, allowing estimation of changing ﬁelds.

The ﬁrst step, equation (20), averages all of the local node values that correspond to a given entry in  $q$ , and is the only step that requires communication. Only nodal values on the boundary of an ROD must be transmitted; nodes on the interior of the ROD directly correspond to a unique entry in  $q$  and need not be communicated.

Equation (21) is the primal update step, and  $\hat{q}_{i}(t)$  is agent ’s local estimate of $i$   $\bar{q}_{i}$ . The third step, equation (22) updates the dual variable  $\nu_{i}(k)$ . Neither the primal nor the dual update steps require communication. When the algorithm converges, the entries of  $\xi_{i}(t)$  and  $\hat{q}_{i}(t)$  converge to the corresponding entries in  $q.$ , so every agent has a piece of the global FEM solution.

The agents cannot compute equation (20) directly because they do not know  $g(i,j)$ . However, if agents  $i$  and  $j$  are not neighbors they do not share nodes and do not contribute to each others’  $\xi_{i}(t)$  update; thus agents only must commu- nicate with their ROD neighbors. Additionally, as long as neighboring agents know the correspondence between the nodes shared with their neighbors, they can compute (20) by averaging all values corresponding to the same node without knowing the global index  $g(i,j)$ . Therefore, agents transmit nodal values on their ROD boundary and some book-keeping information that allows the agents to determine which shared nodes correspond to the same global node.

For example, in the worst case, agents can transmit nodal positions along with nodal values. They can then determine which of their neighbors’ nodes correspond to the same global node. In a more efﬁcient method, agents group their nodal values by ROD edge, labeling each group with the neighboring agent’s identiﬁer and sorting the nodes with in each group in counter-clockwise order relative to the transmitting agent. This technique allows agents to associate the entries in their nodal vector with the nodal values of their neighbors.

Overall, an agent’s memory use increases with the number of nodes in its local mesh. For a ﬁxed number of global nodes, adding more agents decreases the number of nodes per agent (and hence its memory usage); alternatively, adding more agents with ﬁxed capabilities allows the number of global nodes to increase. Thus the agents’ memory usage scales with respect to environment representation.

Communication per agent does not directly depend on the environmental representation size, but rather it increases with the number of nodes on ROD boundaries. For Voronoi regions, as the number of agents increases, the number of Voronoi edges (and hence communication neighbors) per agent remains (on average) constant [20]. As agent density increases, Voronoi region perimeter decreases, and, for ﬁxed , the number of shared nodes per agent approaches the $\ell_{\mathrm{max}}$  number of Voronoi vertices for that agent’s region. Hence, the communication requirements per agent do not increase with increasing agent density. The solutions obtained by individual agents, however, always match the global solution determined over an equivalent global mesh.

## A. Queries

Although agents estimate the ﬁeld within their own RODs, they do not know the ﬁeld outside their ROD. A query system allows a user at a base station to retrieve the estimate at any location by asking any agent. A user can make sparse queries over the whole ﬁeld to obtain a broad estimate and then focus on areas of particular interest. If there are many agents and measurements, making these queries requires signiﬁcantly less data than sending all measurements back to a central computer, a savings that is compounded if communication with a base station is more costly than agent- to-agent communication (e.g., satellite vs. XBee).

Queries consist of the location of the desired estimate. Upon receiving a query, a naive approach employs depth ﬁrst search: if the point is in the agent’s region it returns the estimate, otherwise it forwards the request to one of its neighbors. That neighbor repeats the process, but will only return the request to the initiator after it has exhausted all other neighbors. Since the graph is connected and the agents completely cover the region, an agent with the desired information will eventually be found. This search can usually be made more efﬁcient by always picking the next neighbor based on which is closest to the vector from the current agent to the destination. By alternating between running ADMM iterations until convergence and several rounds of querying, time-varying ﬁelds can be monitored if they change slowly. Algorithm 1 displays the query process from a single agent’s perspective.



Algorithm 1 Query Implementation

1: neighbors ← {(id, ROD edge), ...}

2: (from, location)  $\leftarrow$  receive-query-for-location() 3: if location in ROD then 4:send-estimate-to(from, my-estimate-at(location)) 5: else 6: repeat 7: toedge-closest-to(location, neighbors) $\leftarrow$ 

8: neighbors remove(neighbors, to) $\leftarrow$  9: response send-query(to) $\leftarrow$  10: until has-estimate(response) or is-empty(neighbors) 11: if is-empty(neighbors) then 12:send-not-found-to(from)

13: else 14:send-estimate-to(from, response) 15: end if 16: end if V. SIMULATIONS We simulate 18 agents in an environment given by  over a unit square centered $\bar{\phi}(x,y)=x^{4}y\!+\!x^{3}y\!+\!y^{2}\!+\!x^{2}\!+\!2$  at (0.5, 0.5). The agents have zero mean Gaussian sensor noise with variance of 0.1. Agents use  $\ell_{\mathrm{max}}=0.25$ ,  $\alpha_{0}=0$ , , $\alpha_{1}=0.3$   $\rho=1$ ,  $\mu_{i}=1$  and  $\phi_{0}(x,y)=0$ . Figure 2 depicts the agents’ Voronoi regions, their meshes, their estimates, and a location that the base station queried after running the ADMM algorithm for 8000 time-steps. Figure 3 depicts the actual ﬁeld and Figure 4 shows the error, averaged over 100 Monte-Carlo runs. The distributed solution and standard VIM solution over the same mesh yield the same results. The query was initially directed at agent zero, and the arrows show the chain of queries needed to determine the value at (0.7, 0.6). Due to noise and because no interpolation method is perfect, the actual and estimated values differ.

 



VI. CONCLUSION We have introduced distributed VIM (DVIM), which uses a distributed FEM solver based on ADMM to generate envi- ronment estimates. Although the estimates generated match a centralized solution, no single agent has a full environment representation. The scaling properties of the algorithm mean that adding more agents more densely in the environment does not increase the communication or memory burdens of individual agents. More densely distributed agents, however, provide more detailed and accurate environment estimates. A query system allows a user to determine the environment estimate at any given location. Future work includes devel- oping control laws for mobile DVIM agents, estimating an error ﬁeld from VIM (using the techniques of [4], [6], [21]), and explicitly incorporating time-varying ﬁeld models into the estimation.

 



REFERENCES [1] D. M. Glover, W. J. Jenkins, and S. C. Doney, Marine Science.Cambridge: Cambridge University Press, 2011.



[2] P. Brasseur, J. Beckers, J. Brankart, and R. Schoenauen, “Seasonal temperature and salinity ﬁelds in the mediterranean sea: Climatological analyses of a historical data set,” Deep Sea Research Part I: Oceanographic Research Papers, vol. 43, no. 2, pp. 159 – 192, 1996. [3] C. Troupin, M. Ouberdous, D. Sirjacobs, A. Alvera-Azc´arate, A.Barth,M.-E.Toussaint,S.Watelet,andJ.-M.Beckers, Diva User Guide, GeoHydrodynamics and Environment Research, MARE (GHER), University of Liege, 2015. [Online]. Available: http://modb.oce.ulg.ac.be [4] C. Troupin, A. Barth, D. Sirjacobs, M. Ouberdous, J.-M. Brankart, P. Brasseur, M. Rixen, A. Alvera-Azcrate, M. Belounis, A. Capet, F. Lenartz, M.-E. Toussaint, and J.-M. Beckers, “Generation of analysis and consistent error ﬁelds using the data interpolating variational analysis (diva),” Ocean Modelling, vol. 5253, no. 0, pp. 90 – 101, 2012.
# Environmental Estimation With Distributed Finite Element Agents

Matthew L. ElwinRandy A. FreemanKevin M. Lynch

Abstract— We develop an environmental estimation method that allows large groups of agents to infer the value of an environmental ﬁeld using measurements. Agents maintain estimates for subregions of the domain and communicate with local neighbors, so the method’s communication and memory requirements do not increase with the number of agents, the size of the environment representation, or the agents’ density in the environment. Despite the distributed representation, the union of individual estimates matches an estimate generated by a central computer with access to all measurements employing the variational inverse method, a ﬁnite element-based interpolation procedure. We also introduce a distributed query system, allowing users to determine an estimate anywhere in the domain without accessing all measurements or the full environment representation.

I. INTRODUCTION A common problem in sciences such as oceanography, earth science, and atmospheric science is to infer the value of a scalar ﬁeld (e.g., temperature, chemical concentration, radiation intensity) anywhere in an environment based on irregularly spaced measurements. Typically, the relatively sparse data are processed by a central computer; such meth- ods have been studied extensively (see e.g., [1]). The advent of cheap sensors, however, presents an opportunity to collect more data than ever before. As the number of sensors used to measure an environment increases, so do computation and communication burdens. These problems become especially acute in remote environments such as the ocean, the polar icecaps, outer space, and underground. In such environments, communication with a distant base station may be signiﬁ- cantly harder than communication between agents, making it impractical to send all measurements to a central computer. To address the problem of handling many measurements with limited memory and bandwidth, we develop a dis- tributed version of the variational inverse method (VIM) for data interpolation [2]–[6]. This technique, used in oceanog- raphy, generates an estimate by minimizing a cost functional using the Finite Element Method (FEM) [7]. Rather than having a central computer collect data from sensors, we deploy the sensors on agents that communicate with each other and perform computations. Each agent determines an estimate for a small portion of the environment. No single agent, therefore, stores a full environmental representation



This work is supported by the Ofﬁce of Naval Research, Grant N00014-13-1-0331. The authors are with the Department of Mechanical Engineering (Elwin and Lynch), the Northwestern Institute on Complex Systems (Freeman and Lynch), and the Department of Electrical Engineering and Computer Science (Freeman), Northwestern University, Evanston, IL60208USA.Emails:elwin@u.northwestern.edu, freeman@eecs.northwestern.edu, kmlynch@northwestern.edu.

or all of the measurements. By communicating with local neighbors, however, the collection of all agents’ estimates matches the result obtained by a central computer using VIM. We use the alternating direction method of multipliers (ADMM) to distribute computations and solve the FEM problem [8]. Other methods, which generally require more communication than ADMM include [9] and [10].

We couple our distributed VIM (DVIM) method with a query system, allowing a user to determine the value of the ﬁeld anywhere. The user can obtain a low-resolution view of the overall ﬁeld and then focus on higher resolution estimates in areas of particular interest. Using this system, the full set of measurements and a complete environment representation are never communicated to or stored by a single computer. When communication with a remote base station (e.g., via satellite) costs signiﬁcantly more than inter- agent communication (e.g., via XBee), minimizing such communication becomes especially important.

In Section II we compare DVIM to existing methods in the literature. In Section III we formulate the problem, intro- ducing VIM and FEM. Section IV describes our distributed solution, DVIM, and the query system. Section  $\mathrm{V}$  provides simulation results. We outline our conclusions and future work in Section VI.

II. METHOD COMPARISON We compare DVIM to existing techniques whose com- putation, communication, and memory requirements do not increase with respect to the number of agents. Many criteria can be used to evaluate methods, but in this work we are particularly concerned with three desirable properties: scalability with respect to the environment representation, scalability with respect to agent density, and incorporation of all relevant measurements.



Scalability with respect to the environment representa- tion means that every agent’s memory and communication requirements do not increase with the resolution of the environment model. Methods that scale with the environment representation allow adding more agents with ﬁxed capabil- ities to obtain higher resolution estimates.

Scalability with respect to agent density means that every agent’s memory and communication requirements do not increase as the density of the agents in the environment (and therefore measurement density) increases. High measurement density contributes to more accurate estimates, so methods with this property provide opportunities for improved accu- racy by adding more agents with ﬁxed capabilities.

Methods that incorporate all relevant measurements gen- erate an estimate at a given point using measurements from

ENVIRONMENT REPRESENTATION, SAD - SCALE WITH AGENT DENSITY, IRM - INCORPORATE RELEVANT MEASUREMENTS.

locations correlated with that point. Many methods assume that the correlation between points decreases with distance, and that points further than a characteristic correlation length apart are uncorrelated. Although VIM (and therefore DVIM) does not rely on this assumption (an advantage; it can handle cases when nearby points should be uncorrelated, see [3]), it incorporates all measurements when forming estimates and does have a notion of spatial correlation [4]. Methods that disregard measurements that should inﬂuence an estimate at a given point do not use information efﬁciently.

Two features of all the methods we examine that DVIM currently lacks are control laws for mobile agents and error estimation abilities (we leave these for future work). To ensure a fair comparison, we assume, for all papers, station- ary agents and known error ﬁelds for all agents. Although estimating an error ﬁeld provides beneﬁts, it also imposes additional communication and memory requirements on the agents. Additionally, in [11], movement increases communi- cation and memory requirements because the environmental representation is coupled to the agents’ trajectories.

We now describe several methods and evaluate whether they scale with the environment representation, scale with agent density, and incorporate all relevant measurements. Table I shows this comparison, with more details provided below. Note that a central computer collecting measurements from all agents and applying VIM does not scale with the environment representation or with agent density.

A. Distributed Kalman Filter [12] This method uses a distributed Kalman ﬁlter based on average consensus estimators to incorporate measurements and estimate basis function coefﬁcients that represent the en- vironment. The method could also work using the distributed data fusion techniques of [16]. Communication and memory is proportional to the number of basis functions, which is ﬁxed beforehand; therefore, this method does not scale with the environment representation. Every agent, however, has a full environment estimate. This method scales with agent density because it works over any connected communication graph. The relationship between measurements at different points is incorporated in the basis function choice, so mea- surements contribute to the ﬁeld estimate everywhere.



B. Kriged Kalman Filter [13] This method represents the environment as the mean and covariance functions of a spatial Gaussian process. The representation consists of basis function coefﬁcients (as in [12]) and as a combination of measurements within a given radius  $R$  of each agent. If the coefﬁcients are unknown, communication and memory is proportional to the number of basis functions; however, by assuming known coefﬁcients, communication is independent of the global environment description and this method can scale with the environment representation. Regardless, the communication increases with density: estimation requires the agents to incorporate all measurements within a radius of  $R$ . Since the agents account for measurements within a radius of  $R$  and points further than  $R$  apart are assumed to be uncorrelated, all relevant measurements contribute to the estimate.

 



## C. Non-parametric Information  $I I4J$ 

This method represents the environment as a weighted set of discrete samples from a probability distribution. Accord- ingly, it can approximate any type of environment, not just scalar ﬁelds. Every agent, however, has a complete envi- ronment representation and must communicate information proportional to the size of this representation. This method, therefore, does not scale with the representation size. It does, however, scale with agent density because it works over connected graphs. Information is fused using a particle-ﬁlter like approach, so the estimate at a given location incorporates relevant measurements.

## D. Local Interpolation: Nearest Neighbor [11]

In this method, the environment is a piecewise constant function. When used with stationary agents, an agent’s estimate over its Voronoi region is its measurement. This method, therefore, scales with the environment representation (adding more agents adds more Voronoi cells) and with density. The nearest neighbor approach, however, uses a single measurement to estimate the ﬁeld over each Voronoi cell, so it does not use all relevant measurements. With mobile agents, as originally intended, the initial Voronoi cells are subdivided based on current and past measurements, creating a more detailed estimate. The environment represen- tation, however, is coupled to the agent’s trajectories; lossy compression is used to maintain scalability with respect to the number of measurements when the agents move.

E. Local Interpolation: Inverse Distance [11] Closely related to Nearest Neighbor Local Interpolation, this method uses piecewise continuous functions to represent the environment. Here, measurements have a diminishing effect on the estimate as distance from the measurement de- creases. Thus, unlike the Nearest Neighbor version, estimates at a given point incorporate relevant measurements. However, now the estimate must incorporate all measurements from within a given radius  $R$ , so the method no longer scales with agent density.



##  $F.$  Distributed Finite Element Kalman Filter [15], [17]

In this method, the environment is modeled as a partial differential equation (PDE) in space and time. Using ﬁnite elements over the spatial variables and discretizing over time, the PDE is converted into a discrete time state-space system and a Kalman Filter used to estimate the state. Each agent is responsible for only a subset of the domain; thus the method scales with respect to the environmental model. At each time-step, the state is estimated using the parallel Schwarz method and measurements are incorporated using a Kalman ﬁlter update. The method requires only information shared by adjacent regions to be exchanged and converges to a centralized solution; therefore the method scales with agent density and incorporates all relevant information.



G. Distributed Variational Inverse Method This method, as developed here, is scalable with respect to environment representation, agent density, and incorpo- rates relevant measurements when forming an estimate. The environment representation is the total number of nodes in a global FEM mesh, but each agent handles only a subset of those nodes. As the density of the agents increases, communication costs do not increase because communication is proportional to the number of Voronoi neighbors and the perimeter of every agent’s Voronoi region, which do not increase with density. Finally, in each agents’ region, the estimate matches the centralized VIM method, which incorporates all measurements into the estimate.



Unlike [15], [17], our method does not require a model of the ﬁeld; thus it is applicable in situations when partial differential equations for the environment are not known. Additionally, we present a method to form consistent ﬁnite element meshes in a distributed manner and a query system to extract the estimate, issues that are not addressed in [15], [17].

III. VARIATIONAL INVERSE METHOD

## A. Sensor and Environment Model

The environment is a scalar ﬁeld  $\bar{\phi}(x,y)$  over a region . Stationary agents dispersed over $\Omega\subset\mathbb{R}^{2}$   $\Omega$ , with agent  $i$  located at  $(x_{i},y_{i})\in\Omega$ , inaccurately measure the ﬁeld: of the estimate from measurements. In this paper we use




$$
\begin{array}{c}{{S[\phi]=\displaystyle\left(\frac{\partial^{2}\phi}{\partial x^{2}}\right)^{2}+\displaystyle\left(\frac{\partial^{2}\phi}{\partial y^{2}}\right)^{2}+2\displaystyle\left(\frac{\partial^{2}\phi}{\partial x\partial y}\right)^{2}}}\\ {{\displaystyle+\alpha_{1}\left(\left(\frac{\partial\phi}{\partial x}\right)^{2}+\left(\frac{\partial\phi}{\partial y}\right)^{2}\right)+\alpha_{0}(\phi-\phi_{0})^{2}}}\end{array}
$$


and


$$
D[\phi]=\sum_{k=1}^{n}\mu_{k}\left(\phi\big(x_{k},y_{k}\big)-z_{k}\right)^{2}
$$


(4) where  $\phi_{0}(x,y)$  is a known background ﬁeld,  $\alpha_{1}$  is a weight on the curvature,  $\alpha_{0}$  weights the error of the ﬁeld, and  $\mu_{k}$  weights each measurement.



The known background ﬁeld  $\phi_{0}(x,y)$  is the assumed value of the ﬁeld in the absence of measurements; it allows us to incorporate prior assumptions about the ﬁeld into the estimate. Alternatively,  $\phi_{0}(x,y)$  can be taken to be the average of the data, a linear ﬁt to the data, or zero. When  it makes sense to let $\phi_{0}(x,y)=0$   $\alpha_{0}=0$  to avoid penalizing the magnitude of the resulting estimated ﬁeld.

 $D.$ . Finite Element Method We use FEM to minimize the cost functional (2); details can be found in [7]. FEM consists of the following ﬁve steps: 1) Divide the domain into sub-domains called elements. 2) Find interpolating equations for each element based on values at points within each element, called nodes. 3) Assemble elements into a global system of equations. 4) Solve the equations. 5) Post-process the result to obtain information from the solution, for example, producing a visualization. 1) Dividing the domain: The domain is partitioned into a mesh of  $m$  non-overlapping sub-domains called elements (see Figure 2). Each element’s domain is  $\Omega_{e}$  and the elements approximately cover  $\Omega$ . Elements only intersect at vertices or along full edges. Many element geometries are available. For simplicity, we use a mesh of triangular elements, as gen- erated by the Computational Geometry Algorithms Library (CGAL) [18].

2) Interpolating equations: The estimate within each ele- ment,  $\phi_{e}(x,y)$ , is approximated by a polynomial containing all terms up to order  $d$ , which can be fully described by  $r$  coefﬁcients. This polynomial’s coefﬁcients are related to the value of  $\phi_{e}(x,y)$  at points within the element called nodes:  $\mathbb{R}^{r}$ ere  $w(x,y)$  is the vector of  $r$  shape functions and  $q_{e}\,\in$  Rr is the vector of nodal values. Many shape functions are available. We use quadratic polynomials over triangular domains, so  $r\ \ =\ \ 2$  (see Figure 2). The polynomial is , where $b_{0}\!+\!b_{1}x\!+\!b_{2}y\!+\!b_{3}x^{2}\!+\!b_{4}x y\!+\!b_{5}y^{2}$   $b_{i}$  is a coefﬁcient. There are six coefﬁcients and therefore six nodes (one at each corner and one bisecting each edge).



Each shape function (an element of  $w(x,y))$  corresponds to a node: it is one at that node and zero at all other nodes. For example, with  $r=2$ , evaluating  $w^{T}(x,y)$  at element  $e$ ’s second node yields  $\left[0\quad1\quad0\quad0\quad0\quad0\right]$  1000 0 , which selects the second element of  $q_{e}$  in equation (5).

3) Assembly: Each nodal vector  $q_{e}$  is local to element  $e$ ; however, nodes on adjacent element edges overlap. Gener- ally, therefore, there are  $p<r m$  nodes in the ﬁnite element mesh. The global node vector  $q\ \in\ \mathbb{R}^{p}$ is related to the local node vectors by the Boolean stochastic gather matrices   $L_{e}\in\mathbb{R}^{r\times p}$ such that:


$$
q_{e}=L_{e}q.
$$


(6) By deﬁning when $\phi_{e}(x,y)=0$   $(x,y)\not\in\Omega_{e}$ , an approxi- mation of the whole ﬁeld is




$$
\phi(x,y)\approx\sum_{i=1}^{m}\phi_{e}(x,y).
$$


(7) Note that by using the Dirac delta  $\delta$ , all terms of the cost functional (2) can be written inside the integral as




$$
J[\phi]=\int_{\Omega}\left(S[\phi]+D^{\prime}[\phi]\right)d\Omega,
$$


(8) where




$$
D^{\prime}[\phi]=\sum_{k=1}^{n}\mu_{k}\bigl(\phi-z_{k}\bigr)^{2}\delta\bigl(x-x_{k}\bigr)\delta\bigl(y-y_{k}\bigr).
$$


Substituting equation (7) into the cost functional (9) and performing some manipulations yields


$$
J[\phi]\approx\sum_{e=1}^{m}(q_{e}^{T}K_{e}q_{e}-2q_{e}^{T}g_{e})+\sum_{k=1}^{n}\mu_{k}(z_{k}-\phi_{0}(x_{k},y_{k}))^{2},\eqno(42)
$$


with


$$
\begin{array}{c}{{\displaystyle K_{c}=\int_{\Omega_{c}}\left[\left(\frac{\partial^{2}w}{\partial x^{2}}\right)\left(\frac{\partial^{2}w}{\partial x^{2}}\right)^{T}+\left(\frac{\partial^{2}w}{\partial y^{2}}\right)\left(\frac{\partial^{2}w}{\partial y^{2}}\right)^{T}\right.}}\\ {{\displaystyle\left.+\,2\left(\frac{\partial^{2}w}{\partial x^{2}\partial y}\right)\left(\frac{\partial^{2}w}{\partial x\partial y}\right)^{T}+\alpha_{1}\left(\frac{ 
$$


and


$$
g_{e}=\sum_{(x_{k},y_{k})\in\Omega_{e}}\mu_{k}w(x_{k},y_{k})z_{k}.
$$


(12) Substituting equation (6) into equation (10) and minimiz- ing with respect to  $q$  yields the global FEM equation




$$
K q=g,
$$


(13) where




$$
K=\sum_{e=1}^{m}L_{e}^{T}K_{e}L_{e}
$$


(14) and




$$
g=\sum_{e=1}^{m}L_{e}^{T}g_{e}.
$$


(15) 4) Solving: Solving equation (13) for  $q$  provides the ﬁeld estimate. For brevity we have omitted boundary conditions; implicitly assuming that derivatives of the estimate across the boundary are zero. However, our technique extends to cases when either the derivatives on the boundary or the value of some boundary nodes are speciﬁed.



5) Post-processing: Given the global node vector  $q$ , the ﬁeld can be estimated at any location using equations (7) and (6), allowing us to, for example, plot the estimate.

IV. DISTRIBUTED METHOD Our method, distributed VIM (DVIM), enables agents to estimate ﬁelds using local communication without any agent needing all measurements or a full environment representa- tion. The scaling properties of the algorithm, derived from its distributed nature, allow for more detailed environment rep- resentations as the group size increases without a concomi- tant increase in the memory or communication requirements of individuals. Memory and communication requirements also do not increase with increasing agent density. Individual agents’ estimates are consistent with VIM solved on a central computer with full access to all measurements.



To retrieve estimates from the group requires a query system. When at least one agent can communicate with a base station, users can query ﬁeld estimates at any location. Users may request estimates over a low-resolution grid and then focus on speciﬁc areas, allowing for less communication between the group and the base station.

Agents must perform the following four tasks: 1) Determine a region of dominance (ROD), the area over which they generate estimates. In this paper an agent’s ROD is the agent’s the Voronoi region. 2) Form a mesh within their (ROD) such that the union of all agents’ RODs forms a valid FEM mesh. We modify an existing meshing algorithm for this step. 3) Solve the FEM problem. We use the alternating direc- tion method of multipliers (ADMM) algorithm. 4) Respond to requests directed at determining the es- timate somewhere within the domain. We develop a direction-based depth ﬁrst search query system. These tasks can be repeated indeﬁnitely to provide estimates of time varying ﬁelds.

Determining the ROD (task 1) and forming the mesh (task 2) roughly correspond to the global FEM steps of dividing the domain (step 1) and ﬁnding the interpolating function (step 2). Task 3, where the agents solve the FEM problem, covers both the assembling (step 3) and solving (step 4) steps of global FEM. One advantage of using ADMM over other solving methods is that assembly happens locally on the agents; individual entries in  $K$  and  $g$  are never explicitly constructed by any agent. Task 4, querying, is the distributed version of post-processing (step 5).



We now discuss each step of distributed VIM individually. For clarity, we assume that every agent takes one measure- ment with one sensor, so there are  $n$  agents,  $n$  measurements, and  $n$  sensors; however, this method easily allows agents to take multiple measurements with multiple sensors.

 $I$ ) Region of Dominance: Each agent establishes a region of dominance (ROD)  $\Omega_{i}\,\subset\,\Omega$ , partitioning i $\Omega$ o  $n$  non- overlapping regions that approximately cover Ω. If two agents have intersecting RODs, they are neighbors and must be able to communicate.

WeuseboundedVoronoiregionsforestablishing RODs, deﬁned as the set of all points  $(x,y)$  such that  and $\|(x,y)-(x_{i},y_{i})\|<\|(x,y)-(x_{j},y_{j})\|$   $(x,y)\in\Omega$  for all  $i\neq j$ . Two agents are Voronoi neighbors if their Voronoi regions share an edge. If the agents know their absolute position and can communicate with their Voronoi neighbors, the algorithm of [19] allows distributed computation of the region. As argued in [19], this communication requirement results in scalable networks because, on average, each agent has, at most, six neighbors.

2) Consistent Mesh: Once the agents know their ROD, they must form a mesh consistent with their neighbors’ meshes. Although agents compute their meshes indepen- dently, agents with shared edges must place any mesh vertices on those edges at the same location. Assuming a polygonal ROD, every agent ﬁrst divides its edges into the minimum number of segments such that no segment exceeds some pre-determined maximum length  $\ell_{\mathrm{max}}$ . The vertices subdividing these edges will be vertices in the resulting mesh, and, due to symmetry, they will be at the same absolute location for adjacent agents.

Next, each agent runs the meshing algorithm of [18], which generates a mesh with no triangle edge lengths ex- ceeding  $\ell_{\mathrm{max}}$ . If the algorithm adds vertices on the ROD edges, they must be removed. Vertex removal is well-deﬁned on the structure returned by the meshing algorithm.

3) Distributed FEM: After determining its ROD and meshing it, every agent  $i$  has  $m_{i}$  elements and  $p_{i}$  nodes. Let  denote the set of elements contained in agent $E_{i}$   $i$ ’s ROD. Since elements are not shared between agents,  $E_{i}$  and  $E_{j}$  are disjoint for  $i\neq j$ . Thus, each agent has a regional node vector  $\bar{q}_{i}\;\in\;\mathbb{R}^{p_{i}}$  (analogous to  $q.$ ), which is related to the node vectors for each element in its region by


$$
q_{f}=\bar{L}_{f}\bar{q}_{i},\,\mathrm{for\all\}f\in E_{i},
$$


(16) where the stochastic Boolean matrix  $\bar{L}_{f}$  is analogous to  $L_{e}$ . Likewise, we form the regional matrices and $\bar{K}_{i}$   $\bar{g}_{i}$  anal- ogously to equations (14) and (15) according to




$$
\bar{K}_{i}=\sum_{f\in E_{i}}\bar{L}_{f}^{T}K_{f}\bar{L}_{f}
$$


(17) and




$$
\bar{g}_{i}=\sum_{f\in E_{i}}\bar{L}_{f}^{T}g_{f}.
$$


(18) The map between agent  $i$ ’s node vector and the global node vector is given by  $g(i,j)$  such that  $[\bar{q}_{i}]_{j}$  corresponds to , where $[q]_{g(i,j)}$   $[\cdot]_{j}$  indicates the  $j$ -th entry in a vector.



Using equations (10), (6), (17), and (18), we derive the following constrained minimization problem:


$$
\mathrm{argmin}\;\;\;\sum_{i=1}^{n}\left(\bar{q}_{i}^{T}\bar{K}_{i}\bar{q}_{i}-2\bar{q}_{i}^{T}\bar{g}_{i}\right)
$$


## with respect to  $\bar{q}_{i}$ , for  $i=1$  to n

subject to  $[\bar{q}_{i}]_{j}=[q]_{g(i,j)}$ , for  $i=1\,{\mathrm{to}}\,n$  and  $j=1$  to  $p_{i}$ . (19)

Note that the measurements enter the minimization prob- lem (19) via the term  $\bar{g}_{i}$ . The distributed ADMM method of [8] allows every agent to solve this problem for its own  $\bar{q}_{i}$ . Due to the constraints on  $\bar{q}_{i}$ , the union of all agents solutions is equivalent to solving the global FEM problem on the union of all agents’ meshes.

Let $N_{i,j}$ bethesetofallpairs $(k,l)$ suchthat . In other words, $[\bar{q}_{k}]_{l}=[q]_{g(i,j)}$   $N_{i,j}$  is the set of all agent- index, local-node-index pairs corresponding to the global node index. The cardinality of $g(i,j)$   $N_{i,j}$ ,  $|N_{i,j}|$  is then the number of local nodes corresponding to a given global node. To implement ADMM, every agent  $i$  iterates over three steps. We show a version speciﬁc to our problem derived from [8]:


$$
\begin{array}{l}{{[\xi_{i}(t)]_{j}=\displaystyle\frac{1}{|N_{i,j}|}\sum_{(k,l)\in N_{i,j}}[\hat{q}_{k}(t)]_{l},}}\\ {{\displaystyle\hat{q}_{i}(t+1)=(\bar{K}_{i}+\rho I)^{-1}(\bar{g}_{i}-\nu_{i}(t)+\rho\xi_{i}(t)),}}\\ {{\displaystyle\nu_{i}(t+1)=\nu_{i}(t)+\rho\left(\hat{q}_{i}(t)-\xi_{i}(t)\right),}}\end{array}
$$


where  $\rho>0$  is an optimization parameter,  $t$  is the discrete- time index,, $\xi_{i}(t)~\in~\mathbb{R}^{p_{i}}$   $\hat{q}_{i}(t)\ \in\ \mathbb{R}^{p_{i}}$ and  $\nu_{i}(k)~\in~\mathbb{R}^{p_{i}}$  are the state variables of the ADMM algorithm. The agents execute this algorithm repeatedly until a stopping condition is met; we use a ﬁxed number of iterations, but adopting the stopping criteria of [8] is left for future work. When the algorithm completes, the agents take new measurements and the process begins again, allowing estimation of changing ﬁelds.

The ﬁrst step, equation (20), averages all of the local node values that correspond to a given entry in  $q$ , and is the only step that requires communication. Only nodal values on the boundary of an ROD must be transmitted; nodes on the interior of the ROD directly correspond to a unique entry in  $q$  and need not be communicated.

Equation (21) is the primal update step, and  $\hat{q}_{i}(t)$  is agent ’s local estimate of $i$   $\bar{q}_{i}$ . The third step, equation (22) updates the dual variable  $\nu_{i}(k)$ . Neither the primal nor the dual update steps require communication. When the algorithm converges, the entries of  $\xi_{i}(t)$  and  $\hat{q}_{i}(t)$  converge to the corresponding entries in  $q.$ , so every agent has a piece of the global FEM solution.

The agents cannot compute equation (20) directly because they do not know  $g(i,j)$ . However, if agents  $i$  and  $j$  are not neighbors they do not share nodes and do not contribute to each others’  $\xi_{i}(t)$  update; thus agents only must commu- nicate with their ROD neighbors. Additionally, as long as neighboring agents know the correspondence between the nodes shared with their neighbors, they can compute (20) by averaging all values corresponding to the same node without knowing the global index  $g(i,j)$ . Therefore, agents transmit nodal values on their ROD boundary and some book-keeping information that allows the agents to determine which shared nodes correspond to the same global node.

For example, in the worst case, agents can transmit nodal positions along with nodal values. They can then determine which of their neighbors’ nodes correspond to the same global node. In a more efﬁcient method, agents group their nodal values by ROD edge, labeling each group with the neighboring agent’s identiﬁer and sorting the nodes with in each group in counter-clockwise order relative to the transmitting agent. This technique allows agents to associate the entries in their nodal vector with the nodal values of their neighbors.

Overall, an agent’s memory use increases with the number of nodes in its local mesh. For a ﬁxed number of global nodes, adding more agents decreases the number of nodes per agent (and hence its memory usage); alternatively, adding more agents with ﬁxed capabilities allows the number of global nodes to increase. Thus the agents’ memory usage scales with respect to environment representation.

Communication per agent does not directly depend on the environmental representation size, but rather it increases with the number of nodes on ROD boundaries. For Voronoi regions, as the number of agents increases, the number of Voronoi edges (and hence communication neighbors) per agent remains (on average) constant [20]. As agent density increases, Voronoi region perimeter decreases, and, for ﬁxed , the number of shared nodes per agent approaches the $\ell_{\mathrm{max}}$  number of Voronoi vertices for that agent’s region. Hence, the communication requirements per agent do not increase with increasing agent density. The solutions obtained by individual agents, however, always match the global solution determined over an equivalent global mesh.

## A. Queries

Although agents estimate the ﬁeld within their own RODs, they do not know the ﬁeld outside their ROD. A query system allows a user at a base station to retrieve the estimate at any location by asking any agent. A user can make sparse queries over the whole ﬁeld to obtain a broad estimate and then focus on areas of particular interest. If there are many agents and measurements, making these queries requires signiﬁcantly less data than sending all measurements back to a central computer, a savings that is compounded if communication with a base station is more costly than agent- to-agent communication (e.g., satellite vs. XBee).

Queries consist of the location of the desired estimate. Upon receiving a query, a naive approach employs depth ﬁrst search: if the point is in the agent’s region it returns the estimate, otherwise it forwards the request to one of its neighbors. That neighbor repeats the process, but will only return the request to the initiator after it has exhausted all other neighbors. Since the graph is connected and the agents completely cover the region, an agent with the desired information will eventually be found. This search can usually be made more efﬁcient by always picking the next neighbor based on which is closest to the vector from the current agent to the destination. By alternating between running ADMM iterations until convergence and several rounds of querying, time-varying ﬁelds can be monitored if they change slowly. Algorithm 1 displays the query process from a single agent’s perspective.



Algorithm 1 Query Implementation

1: neighbors ← {(id, ROD edge), ...}

2: (from, location)  $\leftarrow$  receive-query-for-location() 3: if location in ROD then 4:send-estimate-to(from, my-estimate-at(location)) 5: else 6: repeat 7: toedge-closest-to(location, neighbors) $\leftarrow$ 

8: neighbors remove(neighbors, to) $\leftarrow$  9: response send-query(to) $\leftarrow$  10: until has-estimate(response) or is-empty(neighbors) 11: if is-empty(neighbors) then 12:send-not-found-to(from)

13: else 14:send-estimate-to(from, response) 15: end if 16: end if V. SIMULATIONS We simulate 18 agents in an environment given by  over a unit square centered $\bar{\phi}(x,y)=x^{4}y\!+\!x^{3}y\!+\!y^{2}\!+\!x^{2}\!+\!2$  at (0.5, 0.5). The agents have zero mean Gaussian sensor noise with variance of 0.1. Agents use  $\ell_{\mathrm{max}}=0.25$ ,  $\alpha_{0}=0$ , , $\alpha_{1}=0.3$   $\rho=1$ ,  $\mu_{i}=1$  and  $\phi_{0}(x,y)=0$ . Figure 2 depicts the agents’ Voronoi regions, their meshes, their estimates, and a location that the base station queried after running the ADMM algorithm for 8000 time-steps. Figure 3 depicts the actual ﬁeld and Figure 4 shows the error, averaged over 100 Monte-Carlo runs. The distributed solution and standard VIM solution over the same mesh yield the same results. The query was initially directed at agent zero, and the arrows show the chain of queries needed to determine the value at (0.7, 0.6). Due to noise and because no interpolation method is perfect, the actual and estimated values differ.

 



VI. CONCLUSION We have introduced distributed VIM (DVIM), which uses a distributed FEM solver based on ADMM to generate envi- ronment estimates. Although the estimates generated match a centralized solution, no single agent has a full environment representation. The scaling properties of the algorithm mean that adding more agents more densely in the environment does not increase the communication or memory burdens of individual agents. More densely distributed agents, however, provide more detailed and accurate environment estimates. A query system allows a user to determine the environment estimate at any given location. Future work includes devel- oping control laws for mobile DVIM agents, estimating an error ﬁeld from VIM (using the techniques of [4], [6], [21]), and explicitly incorporating time-varying ﬁeld models into the estimation.

 



REFERENCES [1] D. M. Glover, W. J. Jenkins, and S. C. Doney, Marine Science.Cambridge: Cambridge University Press, 2011.



[2] P. Brasseur, J. Beckers, J. Brankart, and R. Schoenauen, “Seasonal temperature and salinity ﬁelds in the mediterranean sea: Climatological analyses of a historical data set,” Deep Sea Research Part I: Oceanographic Research Papers, vol. 43, no. 2, pp. 159 – 192, 1996. [3] C. Troupin, M. Ouberdous, D. Sirjacobs, A. Alvera-Azc´arate, A.Barth,M.-E.Toussaint,S.Watelet,andJ.-M.Beckers, Diva User Guide, GeoHydrodynamics and Environment Research, MARE (GHER), University of Liege, 2015. [Online]. Available: http://modb.oce.ulg.ac.be [4] C. Troupin, A. Barth, D. Sirjacobs, M. Ouberdous, J.-M. Brankart, P. Brasseur, M. Rixen, A. Alvera-Azcrate, M. Belounis, A. Capet, F. Lenartz, M.-E. Toussaint, and J.-M. Beckers, “Generation of analysis and consistent error ﬁelds using the data interpolating variational analysis (diva),” Ocean Modelling, vol. 5253, no. 0, pp. 90 – 101, 2012.
# Crucial information for efficient face searching by humans  and Japanese macaques

Ryuzaburo Nakata $\cdot$ Satoshi Eifuku · Ryoi Tamura

### Received: 3 February 2017 / Revised: 27 November 2017 / Accepted: 30 November 2017   $\copyright$  Springer-Verlag GmbH Germany, part of Springer Nature 2017

Abstract Humans can efficiently detect a face among non-face objects, but few studies of this ability have been conducted in animals.  Here, in Japanese macaques and humans, we examined visual searching for a face and explored what factors contribute to  efficient facial information processing. Subjects were asked to search for an odd target among the different numbers of dis- tracters. Faces of the subjects’ own species, the backs of the head of the subjects’ own species, faces of the subjects’ closely  related species or race, and faces of species that are clearly different from the subjects’ own species were used as the target.  Both the macaques and humans detected a face of their own species more efficiently than a face from a clearly different spe- cies. Similar efficient detections were confirmed for the faces of the subjects’ closely related species or race. These results  suggest that conspecific faces and faces that share morphological similarity with conspecific faces can be detected efficiently  among non-face objects by both humans and Japanese macaques. In another experiment, facial recognition efficiency was  observed when the subjects searched for own-species faces that had lower-spatial-frequency components compared to faces  with higher-spatial-frequency components. It seems reasonable that the ability to search efficiently for faces by using holistic  face processing is derived from fundamental social cognition abilities that are broadly shared among species.

Keywords Faces $^{*}$ Visual search $^{*}$ Old World monkey $\cdot$ Other-race effect $^{*}$ Other-species effect $\cdot$ Holistic face processing

## Introduction

The visual search task, in which a human observer searches  for a target among numerous distracters, is one of the best  methods to explore two different ways of searching and  detecting an object. If the target contains a feature attracting  attention, the observer either searches quickly in an efficient,  parallel manner for a target that “pops out,” or searches  slowly in an inefficient, serial manner (Treisman and Gelade  1980; Wolfe 2001). Numerous studies have reported that an  observer could efficiently detect emotional facial expressions  in a crowd of other faces (Becker et al. 2011). Additionally,  there have been studies in which subjects efficiently searched  for their own faces (Tong and Nakayama 1999) or visually  familiar faces (Hansen and Hansen 1988; Levin 1996; Mon- toute and Tiberghein 2001). Clearly, at least certain limited  types of faces can be sought efficiently from among others  in a visual search task.



In most of these studies, both the target and the distract- ers had similar characteristics. By using a visual search task  with realistic and varied distracters, an appreciable face  “pop-out” effect has been observed. Hershler and Hoch- stein (2005) demonstrated that a real human face pops out  from among varied distracter objects even if more stimuli  are presented simultaneously, and this “pop-out” effect dis- appears when a car, a house, or an animal face is presented  as a target object.

There have been a few studies on visual searching for  faces in non-human primates. We have recently tested visual  searching for faces in Japanese monkeys (Nakata et al. 2014)  and showed that the monkeys efficiently searched upright  face-like stimuli among inverted face-like stimuli as opposed  to searching for inverted face-like stimuli among upright  face-like stimuli. In this study, these face-like stimuli con- sisted of circles with a configuration that was an approxi- mation of the face, and therefore, we could not determine  whether the monkeys indeed recognized face-like-configured  drawings (circles) as realistic, i.e., in a cognitive sense of  “face like.” Certainly, humans can recognize the schematic  drawings as faces, but whether monkeys can remains unset- tled. Visual searching for a realistic face among non-face  objects corresponding to an actual scene is critical for exam- ining this possibility in monkeys. Given that monkeys rec- ognize a schematic face as a member of a facial group, pos- sibly a task using naturalistic stimuli will produce a clearer  pop-out effect. So far, however, a very limited number of  studies of visual searches using realistic faces and varied  distracters have been conducted with non-human primates.  One of the few examples is a series of chimpanzee studies  (Tomonaga and Imura 2015). In that series, chimpanzees  detected conspecific faces more efficiently than they detected  cars and houses. The efficient detection was also shown for  adult human faces, with which the chimpanzees were rich  in visual experience. The underlying mechanisms of facial  recognition may explain the similarity in the processing of  both humans and chimpanzees.

From evidence that Old World monkeys (e.g., baboons  and macaques) demonstrated various facial recognition  abilities (Parr and Hecht 2011) and that our macaque study  (Nakata et al. 2014) showed a pop-out effect during the  search for a schematic face, it is possible that the facial rec- ognition abilities of monkeys follow those of apes. In addi- tion, it is important to more naturally examine visual search  in monkeys by allowing them to freely view the objects.  Macaque studies exploring efficiency in searching for faces  under a free viewing condition (overt search) may provide  greater understanding of the facial processing in a search  task with a natural face and natural objects.

In the present study, we examined the efficient search for  a face by Japanese macaques (Macaca fuscata) and humans  under a free viewing condition. The primary purpose of the  study was to determine the factors involved in the efficient  detection of a face among non-face objects by Japanese  macaques and humans. The study’s secondary purpose was  to explore the extent and the limitations of the facial rec- ognition efficiency of both species. In Experiment 1, we  examined the categorical effect for efficient detection by  using several kinds of natural faces (e.g., conspecific faces  and faces of other-species members). We hypothesized that  each species would exhibit more efficient searches for faces  of their own and similar species’ faces and less efficient  searches for other-species faces that are clearly different  from conspecific faces, given that the search efficiency for  faces is likely to be based on the subjects’ familiarity with  conspecific faces and/or the social relevance of conspecific  faces. In Experiment 2, we further examined which facial  information is important for the pop-out effect by modulat- ing the faces’ spatial frequency components.



## Methods

## Animals

Two Japanese monkeys (M. fuscata; one male, one female;  ages 4–5 years; weight  $4.0{-}7.0\,\mathrm{kg})$  were used for the experi- ments. They were housed alone, one per cage, but allowed  visual and auditory communication with Japanese monkeys  in other cages. Their few human caretakers wore facemasks  and protective clothing to limit the animal’s experience of  whole human faces.

## Human participants

The study’s human participants were 8 university students (2  females, 6 males; ages 19–25 years with an average of 19.8;  all Japanese) having normal or corrected-to-normal vision.

## Stimuli

Target stimuli were 35 colored or gray-scale facial images  chosen from the PrimFace database (http://visiome.neuroinf. jp/primface) or our own face set. One set of 28 faces was  always presented without filtering. The remaining 7 faces  were presented with spatial frequency filtering. A wide  variety of non-face objects from our own stimulus set (480  photographs) were also included as distracter stimuli, after  standardization of their dimensions (for example, see Fig. 1).  Some of the non-face objects were human-made, namely  buildings, clothing, electric appliances, furniture, kitchen  goods, miscellaneous goods, stationery goods, tools, vehi- cles, and food products. Others were natural objects, namely  trees, flowers, rocks, vegetables, fruits, and natural scenes  (Fig. S1 and Table S1). The images were adjusted to the  same standard size ( $.70\times70$ pixels, approximately  $2.0^{\circ}$  of  visual angle). The image processing was carried out with the  use of Adobe Photoshop CS2.

## Apparatus

The monkeys were head-restrained and seated in a primate  chair during the experiment. To record eye movements of  monkeys, a scleral search coil was implanted in one eye. A 

and a Japanese monkey face, respectively, used in Experiment 1. (c,  d) A high-pass filtered human face for human participants and a low- pass filtered monkey face for monkey subjects used in Experiment 2.  Photographs of human faces and objects were prepared by R. Nakata.  Photographs of monkey faces by PrimFace database: http://visiome. neuroinf.jp/primface real-time experimental control system, TEMPO (Reflective  Computing, USA), run on a dedicated personal computer  was used to generate the stimuli, control the behavioral task  events, and record eye movements with a time resolution  of  $4~\mathrm{ms}$ . For humans, we used a keypad to register their  responses. E-Prime® (Psychology Software Tools, Inc., UK)  generated the stimuli, controlled behavioral task events and  recorded data.



## Procedure

### Behavioral task

In both experiments, the subjects searched a face (the target  stimulus) among non-face objects (distracter stimuli). Dur- ing the experiments, subjects could freely view the stimuli  (overt search).

For monkeys, the trials began with the presentation of a  circle at the center of the CRT monitor. After the monkey  fixed its eyes on the circle for an average of  $2000\,\mathrm{ms}$  (range:   $1500–2500\,\mathrm{ms}\mathrm{.}$ ), both target and distracters were simultane- ously presented. The present distracters were selected ran- domly from all distracter stimuli, regardless of the category.  The screen position of each stimulus was selected randomly  from 20 possible positions. If the monkey fixed its eyes on  the target for more than  $1000\,\mathrm{ms}$  (a correct trial), a reinforce- ment ( $0.2\;\mathrm{ml}$  of apple juice) was delivered. Reaction times  were defined as the interval between onset of the search  stimulus array and the starting point of correct fixation on  the target stimuli. After delivery of the reward, there was an  intertrial interval (ITI) of approximately  $10~\mathrm{s}$ on average.  However, if the monkey fixed its eyes on any one of the dis- tracter stimuli for more than  $1000~\mathrm{ms}$  (an error trial), then  the screen blackened, and the ITI began. Error trials were  not rewarded.



The humans’ behavioral task sequence was identical to  that of the monkeys’ in terms of the presentation of the fixa- tion point and search stimulus array, but for the behavioral  response the human participants were required to use a key- pad to indicate when and where the target was presented.  They were required to respond by pressing any key as soon  as possible when they found the target. After that, four num- bered squares were presented on the display. The screen  positions of the four squares corresponded to each of the  four quadrants: upper left, upper right, lower left, and lower  right. The participants were required to identify where the  target was on the four quadrants by pressing the appropriate  key on a keyboard. Accordingly, reaction times were defined  as the interval between onsets of the search stimulus array  and first keypad response.

### Experimental conditions

In Experiment 1, we used 4 types of target stimuli, 7 stimuli  per type, which corresponded to the following 4 conditions:  (1) a frontal view of the face of a member of the subject’s  species or race (Japanese monkey faces for Japanese mon- key subjects and Japanese faces for human participants, all  of whom were Japanese), (2) a back-of-the head view of a  member of the subject’s species (backs of the head), (3) a  front face of a member of the subject’s closely related spe- cies or race (rhesus monkey faces for Japanese monkey sub- jects and Caucasian faces for human participants), and (4) a  front face of a member of a species different from the sub- ject’s species (human faces, Japanese faces, and Caucasian  faces, for monkeys and Macaque faces, Japanese monkey  faces, and rhesus monkey faces for humans).



In Experiment 2, we used 7 frontal face images of dif- ferent individuals from the subject’s own species for target  stimuli. We set 4 experimental conditions, modulating spa- tial frequency content in each face using either of two types  of high-pass cutoff filters,  $(1)>16.8$  cycles/face width ( $\mathrm{c}/\mathrm{\Omega}$  fw) (HPF-16.8 faces) or  $(2)>12.6$ c/fw, or either of two  types of low-pass cutoffs,  $(3)<8.3$  c/fw (LPF-8.3 faces) or   $(4)<4.8$  c/fw. Average stimulus luminance did not differ  according to image type.

### Experimental blocks and sessions

For monkeys, the present study consisted of 2 consecutive  blocks corresponding to the Experiments 1–2. In each block,  we first trained the monkey to search for a target among 3  distracters under 4 different conditions. (The conditions are  described above under (1)–(4) in Experimental conditions.)  In each condition, we trained the monkeys using 3 target  stimuli with one target per trial. The monkeys performed  480 trials per session (4 conditions  $\times3$  target stimuli  $\times\;20$   possible positions on the screen $\times~2$  repetitions) with 1 or  2 sessions per day. This training phase continued until the  monkey achieved  $99\%$  correct performance in 2 consecu- tive sessions. If the monkey achieved this criterion, we next  introduced a test phase: in addition to the same number (i.e.,  3) of distracters as in the training phase, 7, 11, and 19 dis- tracters were used to test the effect of different numbers of  distracters (array sizes of 8, 12, and 20). All methods in  the test phase duplicated the training phase just before the  test phase except for target stimuli, i.e., some of the targets  were new, although they were the same type as those used in  the training phase. Therefore, a block used 28 target stimuli  (12 of the stimuli were used in both the training and test  phases, and 16 were newly used only in the test phases). A  test session consisted of 528 trials (480 duplicates of the  training phase and 48 test trials: 3 array sizes  $\times4$  differ- ent conditions  $\times4$  target stimuli). The test phase continued  for 8 sessions, regardless of the behavioral results for the  particular animal. After the 8th session of the test phase  in the first block (Experiment 1), the initial session in the  training phase of the next block (Experiment 2) began. In  the test trials, the positions of the target faces were limited  to 8 positions selected from 20. Across the 8 sessions of  the test phase, each target appeared once at each position.  Therefore, the average eccentricities of each target in the test  trials were equalized.

Humans participated only in the test phases. There were  384 trials: 240 with an array size of 4 and 144 with the  remaining array sizes. They participated Experiment 1 and  Experiment 2 consecutively in 1 day with a short break  between the two experiments.

## Data analysis

We collected data on the reaction times in each subject  and calculated the means for each condition. We analyzed  the data by conducting an analysis of variance (ANOVA).  For the macaques, we analyzed the data using a two-way  ANOVA with the factors of the array size, i.e., the number  of stimuli (target plus distracters: 8, 12, 20), and the varia- tions in the target object (the four conditions in each experi- ment) by using each session data of the two macaques (eight  sessions each) as statistical subjects. The variations across  sessions were serving to estimate error variance. For the  humans, a two-way ANOVA was used with the factors of the  array size and the variations in the target object. Previous  studies have used the slope of reaction times to search for a  target among an increasing number of distracter items as a  measure of search efficiency (Wolfe 2001), and we therefore  further analyzed the slopes of the mean reaction time as a  function of array size (ms/item) for each data set, using a  linear regression.

## Experiment 1

## Results

### The macaques

The two monkeys required 17 and 30 sessions, respec- tively, to satisfy the  $99\%$  criterion in the training phase. The  mean correct-response rate of the test phase was  $99.48\%$   ( $\mathrm{SE}=0.19\}$ , and there were no significant differences in  the mean correct-response rate between the 4 experimental  conditions. There were also no significant differences in the  mean reaction times between the conditions in the array size  of 4 in this phase (Fig. S2).

We compared the mean reaction times for the three  increased array sizes using new targets in the test phase.  Figure 2a shows the mean reaction times for these condi- tions. The ANOVA revealed significant main effects of the  array size  $(F_{2,30}\,{=}\,10.43,P<0.001$  $P<0.001$ ,  $\eta^{2}\,{=}\,0.06)$  and the vari- ations in target objects ( $(F_{3,45}=3.25$ ,  $P<0.05$ ,  $\eta^{2}=0.05)$ .  Notably, a significant interaction between these two factors   $\eta^{2}=0.07)$  $(F_{6,90}=2.70,P<0.05,\eta^{2}=0.07)$  was also revealed. With  respect to the array size, a simple main effect was confirmed  for human faces  $(F_{2,30}=10.89$ ,  $P<0.001$ ,  $\eta^{2}=0.66)$  and  backs of the head  $(F_{2,30}=5.37$ ,  $P<0.01$ ,  $\eta^{2}=0.33)$ , but  not for the faces of the two species of monkey (Japanese  macaque and rhesus macaque).



The analysis of the slopes of the mean reaction time as a  function of the array size (Fig. 2a) showed flatter gradients  for the Japanese macaque faces ( $(-~1.68~\mathrm{ms}$  per item) and  rhesus macaque faces  $(-~0.09~\mathrm{ms}$  per item) compared to  those for the backs of the head (38.12 ms per item) and the  human faces  $\mathrm{\Delta}54.66\:\mathrm{ms}$  per item).

### The humans

The mean correct-response rate in the test phase was  $99.71\%$    $(\mathrm{SE}=0.14)$ , and there were no significant differences in this  rate between the 4 experimental conditions. There were also  no significant differences in the mean reaction times between  the conditions with the array size of 4 in this phase (Fig. S1).

Figure 2b shows the mean reaction times for the three  increased array sizes. The ANOVA revealed significant main  effects of the array size  $(F_{2,14}=8.97$ ,  $P<0.01$ ,  $\eta^{2}=0.09)$   and the variations in target objects  $(F_{3,21}\,=\,34.291,$ ,  ,  $P<0.001$  $\eta^{2}=0.43)$ , and a significant interaction between  these two factors  $(F_{6,42}=2.67$ ,  $P<0.05$ ,  $\eta^{2}=0.08)$ . With  respect to the array size, a simple main effect was confirmed  for Macaque faces  $(F_{2,30}=7.84\$ ,  $P<0.001$ ,  $\eta^{2}=0.45)$  and  backs of the head  $(F_{2,30}=9.24.$ ,  $P<0.001$ ,  $\eta^{2}=0.53)$ , but  not for the human faces (both Japanese and Caucasian faces).

The analysis of the slopes of the mean reaction time as a  function of the array size (Fig. 2b) showed flatter gradients  for the Japanese faces  $(-~0.61$  ms per item) and Caucasian  faces ( $7.54\,\mathrm{ms}$  per item) compared to those for the backs of  the head  $(43.70\,\mathrm{ms}$  per item) and Macaque faces  $(39.82\:\mathrm{ms}$  per item).

## Discussion

We tested the Japanese macaques’ and human participants’  face searching ability by administering similar tasks. The  realistic faces were searched efficiently among varied dis- tracters by monkeys. These results replicated the previous  finding (Nakata et al. 2014) in which monkeys searched for  a face-like drawing among non-varied distracters, by using  more natural settings. No significant increases in the reac- tion times for the increase in the array sizes were observed  when the frontal face of own genus for the monkeys was  searched by the Japanese macaques and the humans. It thus  seems reasonable that not only humans and chimpanzees  but also Japanese macaques detect the frontal-view faces  of conspecifics efficiently. The range of distracter objects  presented in this experiment was exceedingly large, and the  monkeys’ visual experience of some of these distracters  might be different from that of the human participants. In  other words, the monkeys might not have encountered most  of the human-made objects in their lives, whereas human  participants presumably had sufficient experience with all  presented objects. Nevertheless, the pattern of results was  almost the same for humans and monkeys. Therefore, the  difference in daily experiences for some of the distracter  objects did not appear to affect the present results.



The faces used here, i.e., both the two types of human  faces (Japanese and Caucasian) and the two types of  Macaque faces (Japanese macaque and rhesus macaque),  share several facial features in terms of function and shape,  and the arrangement of the facial features is largely com- mon. These similarities between human faces and Macaque  faces — in terms of sensitivity to the first-order spatial con- figuration among facial features (Diamond and Carey 1986)  as “a face” — were less effective for the efficient detection  of the faces. In contrast, the information from faces that is  used to infer “a species” was effective for the detection. The  macaques in this experiment detected the macaque faces  more efficiently than they detected the human faces, and the  human participants detected human faces more efficiently  than the macaque faces. These results concur with the results  of Experiment 4 in the above-mentioned study of chimpan- zees (Tomonaga and Imura 2015) in which the chimpanzees  did not efficiently detect faces of Japanese macaques.

The advantage in the recognition of congeneric faces for  monkeys as well as conspecific faces for humans has been  referred to as own- or other-species effect (OSE) in behav- ioral studies (Campbell et al. 1997; Pascalis and Bachevalier  1998; Dufour et al. 2006; Scott and Fava 2013) and neurosci- ence studies (de Haan et al. 2002; Scott et al. 2005; Sigala  et al. 2011). Our present findings suggest a robust OSE on  the efficient detection of faces. We observed that the efficient  detection of a face is likely to occur not toward every face  stimulus, which shared the first-order spatial configuration,  but toward congeneric face stimuli which may be at a basic  and entry level classification (Tarr and Cheng 2003) of the  description of faces.

Numerous studies indicate that faces from a human sub- ject’s own race are recognized or discriminated better than  faces of humans of other races, known as the other-race  effect (ORE) in humans (Hugenberg et al. 2010; Anzures  et al. 2013; O’Toole and Natu 2013), which is similar to  the OSE. Japanese macaques clearly discriminated pictures  of their own species from those of closely related species  (Fujita 1987), indicating that Japanese macaques scanned  other macaques in a manner that is similar to that observed  in humans who are subject to the ORE.

In the present study, in contrast to the ORE confirmed in  previous studies, the human subjects (who were Japanese)  were able to efficiently detect the faces of another race (Cau- casian) among non-face objects. It is likely that the human  participants had experienced seeing many Caucasian faces  in real life or through the internet, although Japanese have  less experience with Caucasian faces in general. It must be  noted that Japanese faces were detected just slightly faster  than Caucasian faces. This may indicate that the Japanese  participants processed Caucasian faces as having very subtle  differences in the configuration of facial features compared  to faces of their own ethnicity. The Japanese macaques  were also observed to efficiently detect the faces of another  (macaque) species, the rhesus faces. The present results sug- gest that, contrary to the OSE, the ORE has little impact  on the efficient detection of the faces at the basic level of  classification. Thus, subtle information that is useful for the  discrimination of race is likely to be ignored when search- ing a face among non-face objects, and each face would be  processed similarly as a face.



Stronger holistic or configural processing underlies the  own-race advantage in face memory, in humans (Tanaka  et al. 2004; Michel et al. 2006) and in chimpanzees (Tomon- aga and Imura 2015). In addition, studies with human sub- jects obtained convincing evidence of the importance of  high-level, holistic face characteristics (Maurer et al. 2002;  Burke and Sulikowski 2013) for the efficient detection of  conspecific faces (Hershler and Hochstein 2005). Consid- ering that Old World monkeys use a holistic or configural  process when they recognize the differences between con- specific faces (Martin-Malivel et al. 2006; Adachi et al.  2009; Dahl et al. 2009; Gothard et al. 2009), it is likely that  the efficient detection confirmed in the present experiment  was due to a process based on the holistic information of  own-class faces rather than a process based on distinctive  facial parts.

However, in another human study (VanRullen 2006), the  pop-out effect similar to that observed in the above human  study (Hershler and Hochstein 2005) was confirmed, but it  was not limited to faces, was not holistic, and was due to  low-level properties of the images. Although the pop-out  effect of faces from a variety of heterogeneous distracters  in humans is now widely accepted, there is still room for a  considerable measure of disagreement about what informa- tion included in the face is important for the pop-out effect  (Hershler and Hochstein 2006). Therefore, in Experiment 2,  we further examined which facial information (holistic high- level information of faces or low-level visual information  including faces) is important for the pop-out effect.

## Experiment 2

## Results

### The macaques

In the training phase, the two macaques required five and  eight sessions, respectively, to satisfy the  $99\%$  criterion.  The mean correct-response rate in the subsequent test  phase was  $99.13\%$  ( $\mathrm{SE}=0.28)$ , and there were no sig- nificant differences in the rate between the experimental  conditions. There were also no significant differences in  the mean reaction times between the conditions with the  array size of 4 in this phase (Fig. S1).



Figure 3a shows the mean reaction times for the three  increased array sizes in the test phase. The ANOVA  revealed significant main effects of the array size  ,  $(F_{2,30}=25.07\$  $P<0.001$ ,  $\eta^{2}=0.08)$  and the variation  in target objects  $(F_{3,45}=17.91\$ ,  $P<0.001$ ,  $\eta^{2}=0.14)$   and a significant interaction between these two factors  ,  $(F_{6,90}=4.52$  $P<0.001$ ,  $\eta^{2}=0.09)$ . The simple main  effect of the array size was confirmed only for the HPF- 16.8 faces  $(F_{2,30}\,{=}\,30.08,P<0.001,\eta^{2}\,{=}\,0.84)$  $\eta^{2}=0.84)$ ; there were  no significant differences between the array sizes for each  of the other three types of faces.

Our analysis of the slopes of the mean reaction time  as a function of the array size showed a steeper gradient   $(34.43~\mathrm{ms}$  per item) for the HPF-16.8 faces compared to  the two low-pass filtered faces (LPF-8.3 faces,  $5.69~\mathrm{ms}$   per item; LPF-4.8 faces,  $8.64~\mathrm{ms}$  per item). The slope for  the HPF-12.6 faces (9.99 ms per item) was between these  steep and flat gradients.

### The humans

The mean correct-response rate in the test phase was  $99.64\%$   ( $(\mathrm{SE}=0.11)$ , and there were no significant differences in this  rate between the experimental conditions. There were also  no significant differences in the mean reaction times between  the conditions with the array size of 4 in this phase.

Figure 3b shows the mean reaction times for the three  increased array sizes in the test phase. The ANOVA revealed  significant main effects of the array size  $(F_{2,14}=9.27$ ,  ,  $P<0.01$  $\eta^{2}=0.12)$  and the variation in target objects   $\eta^{2}=0.34)$  $(F_{3,21}=18.80,P<0.001,\eta^{2}=0.34)$ , and a significant inter- action between these two factors  $(F_{6,42}=3.12$ ,  $P<0.05$ ,  . With respect to the array size effect, the simple  $\eta^{2}=0.08)$  main effect was confirmed for two types of HPF faces (HPF- 16.8 faces, $F_{2,30}\,{=}\,14.52,P\,{<}\,0.001,\eta^{2}\,{=}\,0.70$  $\eta^{2}\,{=}\,0.70$  and HPF-12.6  faces,  $F_{2,30}=5.48$ ,  $P<0.01$ ,  $\eta^{2}=0.26)$ . There were no  significant differences between the array sizes for the two  types of LPF faces.

Our analysis of the slopes of the mean reaction time  as a function of the array size showed steeper gradients  ( $30.30~\mathrm{ms}$  per item) for the HPF-16.8 faces and the HPF- 12.6 faces (21.81 ms per item) compared to the two types  of low-pass filtered faces (LPF-8.3 faces, 6.90 ms per item;  LPF-4.8 faces, 4.91 ms per item).

## Discussion

In the present experiment, when the array size was increased,  both the humans and macaques showed almost no or only  slight increases in reaction times when searching for the  LPF faces, whereas they showed steeper slopes for the HPF  faces; in particular, the slope for the HPF-16.8 faces showed  a clearly different trend compared to that for the LPF faces.  These results suggest that the information provided by low- spatial-frequency components in faces is more important  than the information provided by high-spatial-frequency  components for efficient facial searching among non-face  objects. This may also be true when naturalistic own-class  faces are presented as the target, as in Experiment 1.

Previous studies have shown that high-spatial-frequency  components provide information about distinct facial parts  relatively well and that low-spatial-frequency components  provide holistic information (Vuilleumier et al. 2003; Gof- faux and Rossion 2006). Actually, the LPF faces appeared  blurred, while the HPF faces looked very sharp and each  facial part appeared distinctly. Therefore, in searching for  the LPF faces, it was difficult for the subject to focus on  the individual parts of the face, and this difficulty may have  augmented the process based on holistic information. In  contrast, in searching for the HPF faces, it was easier for the  subject to focus on the individual parts of the face, which  may have augmented the process based on local informa- tion from individual facial parts. Our findings thus suggest  that the process based on holistic information which is not  ascribed to individual facial parts is more critical for effi- cient face searching than the process based on local informa- tion. In this experiment, we did not use intact faces in which  the spatial frequency contents were not modulated. It is thus  debatable whether the efficient facial searching for the LPF  faces is similar to the facial searching of intact faces (e.g.,  the own-species faces in Experiment 1). Further research is  needed to address this issue.

## General discussion

Our findings have two important implications. First, the  results for the humans and Japanese macaques showed  similar trends at most points in both experiments, suggest- ing that both species possess a mechanism that allows them  to search for a face among non-face objects in an efficient  manner. Some studies claimed that unlike chimpanzees, rhe- sus monkeys did not show the Thatcher illusion (Thomp- son 1980), a measure of face processing ability in which  changes in the configuration of facial features are difficult  to detect in an inverted face (Weldon et al. 2013). There  may be potential evolutionary differences in face processing  among primates. However, other studies have systematically  shown that monkeys perceive the Thatcher illusion (Adachi  et al. 2009; Dahl et al. 2010). Furthermore, the Thatcher  illusion was shown even in squirrel monkeys, a New World  monkey that shows great differences from humans in terms  of ecology, morphology, and anatomy (Nakata and Osada  2012). Other studies also did not find the differences in face  processing among primates (Dahl et al. 2013). Therefore,  it seems reasonable that the ability to search efficiently for  faces derives from fundamental social cognition abilities  broadly shared among primate species, although there is  room for further investigation to determine whether other  species showing social behavior than humans (Hershler and  Hochstein 2005), chimpanzees (Tomonaga and Imura 2015)  and Japanese macaques can efficiently search for a realistic  face among various non-face objects; for example, among  New World monkeys whose face recognition abilities are  somewhat similar to those of humans (Weiss et al. 2001;  Neiworth et al. 2007; Nakata and Osada 2012).



Second, efficient face detection was confirmed when  the searched-for faces had lower-spatial-frequency compo- nents and when the searched-for faces shared similar vis- ual description as “own-species faces” (e.g., faces of both  Japanese and Caucasians for the human participants and  macaque faces of both Japanese and rhesus macaques for  the monkey subjects). The slope of the increase in reaction  time to detect the target among additional distracter items  was used as a measure of search efficiency in human studies,  with a slope close to zero in efficient parallel search (Wolfe  2001). Our results showed this trend relatively well in the  searching of conspecifics faces and LPF faces. These results  suggest that (1) processing based on the holistic information  of the faces is important for the efficient detection of faces;  in other words, the holistic processing of the faces is clearly  meaningful in the earlier stage of visual processing as well  as in the later stages; and (2) the processing that is developed  due to sufficient experience with conspecific faces could be  generalized over the extent of the experience.

Under our experimental settings, why did humans and  monkeys conduct searches for the same face in different  manners? For example, why did humans search for a human  face efficiently, while monkeys searched for the same face  inefficiently? Viewed in this light, the most likely explana- tion is that, in both humans and Japanese macaques, the  faces for which the subjects searched efficiently became  specialized through perceptual narrowing in early life (Pas- calis et al. 2002) and/or daily experience in later years. The  specialization may be endowed innately or it may occur at  a very early stage of life as a study indicated that newborn  monkeys can acquire the knowledge of basic structure of  own-species faces (Sugita 2009). Such specialization would  lead to an increase in useful information (i.e., holistic infor- mation), which would further augment the efficiency of the  search.

Morphological similarities among frequently encountered  faces and infrequently encountered ones (e.g., faces of sub- jects’ closely related species) make use of similar configural  processing in the faces (Dahl et al. 2014). Regarding our pre- sent results, it is thus likely that differences in the efficient  detection of the face classes and the differences between  the OSE and the ORE in efficient face detection are due to  the differences in the morphological similarity among face  classes.

In regard to the main conclusion of this study, our results  strongly suggest that conspecific faces were detected effi- ciently among non-face objects due to a process in which  the face recognition mechanism is shared not only among  humans but also among at least one species of Old World  monkey, i.e., Japanese macaques.

Acknowledgments The authors would like to thank Mr. Yoshio Dai- mon and Mr. Takashi Kitamura for their technical assistance and  care for animals. The face images of monkeys used in this study are  provided by PrimFace database: http://visiome.neuroinf.jp/primface,  funded by Grant-in-Aid for Scientific Research on Innovative Areas,  “Face Perception and Recognition” from Ministry of Education, Cul- ture, Sports, Science, and Technology (MEXT), Japan.

Funding This study was supported by JSPS KAKENHI Grant Numbers  JP26350992, JP17K01991, JP23500465 and JP26350992) and a Grant- in-Aid for Scientific Research on Innovative Areas, “Face Perception  and Recognition” (No. 20119006) from Ministry of Education, Culture,  Sports, Science and Technology (MEXT), Japan.

## Compliance with ethical standards 

Conflict of interest All authors declare that there is no conflict of in- terest.

Human and animal rights The experimental protocols for animals were  approved by The Animal Care and Use Committee of the University  of Toyama (Permit  $\#$ A2013MED-61). All experimental protocols  conformed to the National Institutes of Health (NIH) guidelines for  the humane care and use of laboratory animals and to the Association  for Research in Vision and Ophthalmology statement for use of ani- mals in ophthalmic and vision research. The experimental protocols  for human participants accorded with the Declaration of Helsinki and  were approved by the Research Ethics Committees of the University of  Toyama (Permit #s 130 and 24-41). The human participants provided  written informed consent after an explanation of the study’s purpose.

## References

Adachi I, Chou DP, Hampton RR (2009) Thatcher effect in mon- keys demonstrates conservation of face perception across pri- mates. Curr Biol 19:1270–1273. https://doi.org/10.1016/j. cub.2009.05.067 Anzures G, Quinn PC, Pascalis O, Slater AM, Tanaka JM, Lee K (2013)  Developmental origins of the other-race effect. Curr Dir Psychol  Sci 22:173–178. https://doi.org/10.1177/0963721412474459 Becker DV, Anderson US, Mortensen CR, Neufeld SL, Neel R (2011)  The face in the crowd effect unconfounded: happy faces, not angry  faces, are more efficiently detected in single- and multiple-target  visual search tasks. J Exp Psychol Gen 140:637–659. https://doi. org/10.1037/a0024060 Burke D, Sulikowski D (2013) The evolution of holistic process- ing of faces. Front Psychol 4:11. https://doi.org/10.3389/ fpsyg.2013.00011 Campbell R, Pascalis O, Coleman M, Wallace SB, Benson PJ (1997)  Are faces of different species perceived categorically by human  observers? Proc R Soc Lond B 264:1429–1434. https://doi. org/10.1098/rspb.1997.0199 Dahl CD, Wallraven C, Bülthoff HH, Logothetis NH (2009) Humans  and macaques employ similar face-processing strategies. Curr  Biol 19:509–513. https://doi.org/10.1016/j.cub.2009.01.061 Dahl CD, Logothetis NK, Bülthoff HH, Wallraven C (2010) The  Thatcher illusion in humans and monkeys. Proc Biol Sci  277:2973–2981. https://doi.org/10.1098/rspb.2010.0438 Dahl CD, Rasch MJ, Tomonaga M, Adachi I (2013) Development pro- cesses in face perception. Sci Rep 3:1044. https://doi.org/10.1038/ srep01044 Dahl CD, Rasch MJ, Chen CC (2014) The other-race and other-species  effects in face perception—a subordinate-level analysis. Front  Psychol 5:1068. https://doi.org/10.3389/fpsyg.2014.01068 de Haan M, Pascalis O, Johnson MH (2002) Specializa- tion of neural mechanisms underlying face recognition in  human infants. J Cogn Neurosci 14:199–209. https://doi. org/10.1162/089892902317236849 Diamond R, Carey S (1986) Why faces are and are not special: an  effect of expertise. J Exp Psychol Gen 115:107–117. https://doi. org/10.1162/089892902317236849 Dufour V, Pascalis O, Petit O (2006) Face processing limitations to  own species in primates: a comparative study in brown capuchins,  Tonkean macaques and humans. Behav Process 73:107–113.  https://doi.org/10.1016/j.beproc.2006.04.006 Fujita K (1987) Species recognition by five macaque monkeys. Pri- mates 28:353–366. https://doi.org/10.1007/BF02381018 Goffaux V, Rossion B (2006) Face are “spatial”–holistic face  perception is supported by low spatial frequencies. J Exp  Psychol Hum Percept Perform 32:1023–1039. https://doi. org/10.1037/0096-1523.32.4.1023 Gothard KM, Erickson CA, Amaral DG (2009) How do rhesus  monkeys (Macaca mulatta) scan faces in a visual paired com- parison task? Anim Cogn 7:25–36. https://doi.org/10.1007/ s10071-003-0179-6 Hansen C, Hansen R (1988) Finding the face in the crowd: an anger  superiority effect. J Pers Soc Psychol 54:917–924. https://doi. org/10.1037/0022-3514.54.6.917 Hershler O, Hochstein S (2005) At first sight: a high-level pop out effect  for faces. Vision Res 45:1707–1724. https://doi.org/10.1016/j. visres.2004.12.021 Hershler O, Hochstein S (2006) With a careful look: still no low-level  confound to face pop-out. Vision Res 46:3028–3035. https://doi. org/10.1016/j.visres.2006.03.023 Hugenberg K, Young SG, Bernstein MJ, Sacco DF (2010) The cat- egorization-individualization model: an integrative account of  the other race recognition deficit. Psychol Rev 117:1168–1187.  https://doi.org/10.1037/a0020463 Levin DT (1996) Classifying faces by race: the structure of face cat- egories. J Exp Psychol Learn Mem Cogn 22:1364–1382. https:// doi.org/10.1037/0278-7393.22.6.1364 Martin-Malivel J, Mangini MC, Fagot J, Biederman I (2006) Do  humans and baboons use the same information when categoriz- ing human and baboon faces? Psychol Sci 17:599–607. https:// doi.org/10.1111/j.1467-9280.2006.01751.x Maurer D, Le Grand R, Mondloch CJ (2002) The many faces of  configural processing. Trends Cogn Sci 6:255–260. https://doi. org/10.1016/S1364-6613(02)01903-4 Michel C, Caldara R, Rossion B (2006) Same-race faces are perceived  more holistically than other-race faces. Vis Cogn 14:55–73.  https://doi.org/10.1111/j.1467-9280.2006.01752.x Montoute T, Tiberghein G (2001) Unconscious familiarity and local  context effects on low-level face processing: a reconstruction  hypothesis. Conscious Cogn 10:503–523. https://doi.org/10.1006/ ccog.2001.0524 Nakata R, Osada Y (2012) The Thatcher illusion in squirrel mon- keys (Saimiri sciureus). Anim Cogn 15:517–523. https://doi. org/10.1007/s10071-012-0479-9 Nakata R, Eifuku S, Tamura R (2014) Effects of tilted orientations and  face-like configurations on visual search asymmetry in macaques.  Anim Cogn 17:67–76. https://doi.org/10.1007/s10071-013-0638-7 Neiworth JJ, Hassett JM, Sylvester CJ (2007) Face processing in  humans and new world monkeys: the influence of experimen- tal and ecological factors. Anim Cogn 10:125–134. https://doi. org/10.1007/s10071-006-0045-4 O’Toole AJ, Natu V (2013) Computational perspectives on the other- race effect. Vis Cogn 21:1121–1137. https://doi.org/10.1080/13 506285.2013.803505 Parr LA, Hecht EE (2011) Face perception in non-human primates.  In: Calder AJ, Rhodes G, Johnson MH, Haxby JV (eds) The  oxford handbook of face perception, 1st edn. Oxford University  Press, New York, pp 691–706. https://doi.org/10.1093/oxfor dhb/9780199559053.013.0035 Pascalis O, Bachevalier J (1998) Face recognition in primates: a cross- species study. Behav Process 43:87–96. https://doi.org/10.1016/ S0376-6357(97)00090-9 Pascalis O, de Haan M, Nelson CA (2002) Is face processing species- specific during the first year of life? Science 296:1321–1323.  https://doi.org/10.1126/science.1070223 Scott LS, Fava E (2013) The own-species face bias: a review of devel- opmental and comparative data. Vis Cogn 21:1364–1391. https:// doi.org/10.1080/13506285.2013.821431 Scott LS, Shannon RW, Nelson CA (2005) Behavioral and electro- physiological evidence of species-specific face processing. Cogn  Affect Behav Neurosci 5:405–416. https://doi.org/10.3758/ CABN.5.4.405 Sigala R, Logothetis NK, Rainer G (2011) Own-species bias in the  representations of monkey and human face categories in the  primate temporal lobe. J Neurophysiol 105:2740–2752. https:// doi.org/10.1152/jn.00882.2010 Sugita Y (2009) Innate face processing. Curr Opin Neurobiol 19:39– 44. https://doi.org/10.1016/j.conb.2009.03.001 Tanaka JW, Kiefer M, Bukach CM (2004) A holistic account of the  own-race effect in face recognition: evidence from a cross- cultural study. Cognition 93:B1–B9. https://doi.org/10.1016/j. cognition.2003.09.011 Tarr MJ, Cheng YD (2003) Learning to see faces and objects.  Trends Cogn Sci 7:23–30. https://doi.org/10.1016/ S1364-6613(02)00010-4 Thompson P (1980) Margaret Thatcher: a new illusion. Perception  9:483–484. https://doi.org/10.1068/p090483 Tomonaga M, Imura T (2015) Efficient search for a face by chimpan- zees (Pan troglodytes). Sci Rep 5:11437. https://doi.org/10.1038/ srep11437 Tong F, Nakayama K (1999) Robust representations for faces: evi- dence from visual search. J Exp Psychol Hum Percept Perform  25:1016–1035. https://doi.org/10.1037//0096-1523.25.4.1016 Treisman AM, Gelade G (1980) A feature-integration the- ory of attention. Cogn Psychol 12:97–136. https://doi. org/10.1016/0010-0285(80)90005-5 VanRullen R (2006) On second glance: still no high-level pop-out effect  for faces. Vision Res 46:3017–3027. https://doi.org/10.1016/j. visres.2005.07.009 Vuilleumier P, Armony JL, Driver J, Dolan R (2003) Distinct spatial  frequency sensitivities for processing faces and emotional expres- sions. Nat Neurosci 6:624–631. https://doi.org/10.1038/nn1057 Weiss DJ, Kralik JD, Hauser MD (2001) Face processing in cotton- top  tamarins (Saguinus oedipus). Anim Cogn 3:191–205. https://doi. org/10.1007/s100710000076 Weldon KB, Taubert J, Smith CL, Parr LA (2013) How the Thatcher  illusion reveals evolutionary differences in the face processing  of primates. Anim Cogn 16:691–700. https://doi.org/10.1007/ s10071-013-0604-4 Wolfe JM (2001) Asymmetries in visual search: an introduction.  Percept Psychophys 63:381–389. https://doi.org/10.3758/ BF03194406

                                              

                                             

                                            

                                           

                                          

                                         

                                        

                                       

                                      

                                     

                                    

                                   

                                  

                                 

                                

                               

                              

                             

                            

                           

                          

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 


# Crucial information for efficient face searching by humans  and Japanese macaques

Ryuzaburo Nakata $\cdot$ Satoshi Eifuku · Ryoi Tamura

### Received: 3 February 2017 / Revised: 27 November 2017 / Accepted: 30 November 2017   $\copyright$  Springer-Verlag GmbH Germany, part of Springer Nature 2017

Abstract Humans can efficiently detect a face among non-face objects, but few studies of this ability have been conducted in animals.  Here, in Japanese macaques and humans, we examined visual searching for a face and explored what factors contribute to  efficient facial information processing. Subjects were asked to search for an odd target among the different numbers of dis- tracters. Faces of the subjects’ own species, the backs of the head of the subjects’ own species, faces of the subjects’ closely  related species or race, and faces of species that are clearly different from the subjects’ own species were used as the target.  Both the macaques and humans detected a face of their own species more efficiently than a face from a clearly different spe- cies. Similar efficient detections were confirmed for the faces of the subjects’ closely related species or race. These results  suggest that conspecific faces and faces that share morphological similarity with conspecific faces can be detected efficiently  among non-face objects by both humans and Japanese macaques. In another experiment, facial recognition efficiency was  observed when the subjects searched for own-species faces that had lower-spatial-frequency components compared to faces  with higher-spatial-frequency components. It seems reasonable that the ability to search efficiently for faces by using holistic  face processing is derived from fundamental social cognition abilities that are broadly shared among species.

Keywords Faces $^{*}$ Visual search $^{*}$ Old World monkey $\cdot$ Other-race effect $^{*}$ Other-species effect $\cdot$ Holistic face processing

## Introduction

The visual search task, in which a human observer searches  for a target among numerous distracters, is one of the best  methods to explore two different ways of searching and  detecting an object. If the target contains a feature attracting  attention, the observer either searches quickly in an efficient,  parallel manner for a target that “pops out,” or searches  slowly in an inefficient, serial manner (Treisman and Gelade  1980; Wolfe 2001). Numerous studies have reported that an  observer could efficiently detect emotional facial expressions  in a crowd of other faces (Becker et al. 2011). Additionally,  there have been studies in which subjects efficiently searched  for their own faces (Tong and Nakayama 1999) or visually  familiar faces (Hansen and Hansen 1988; Levin 1996; Mon- toute and Tiberghein 2001). Clearly, at least certain limited  types of faces can be sought efficiently from among others  in a visual search task.



In most of these studies, both the target and the distract- ers had similar characteristics. By using a visual search task  with realistic and varied distracters, an appreciable face  “pop-out” effect has been observed. Hershler and Hoch- stein (2005) demonstrated that a real human face pops out  from among varied distracter objects even if more stimuli  are presented simultaneously, and this “pop-out” effect dis- appears when a car, a house, or an animal face is presented  as a target object.

There have been a few studies on visual searching for  faces in non-human primates. We have recently tested visual  searching for faces in Japanese monkeys (Nakata et al. 2014)  and showed that the monkeys efficiently searched upright  face-like stimuli among inverted face-like stimuli as opposed  to searching for inverted face-like stimuli among upright  face-like stimuli. In this study, these face-like stimuli con- sisted of circles with a configuration that was an approxi- mation of the face, and therefore, we could not determine  whether the monkeys indeed recognized face-like-configured  drawings (circles) as realistic, i.e., in a cognitive sense of  “face like.” Certainly, humans can recognize the schematic  drawings as faces, but whether monkeys can remains unset- tled. Visual searching for a realistic face among non-face  objects corresponding to an actual scene is critical for exam- ining this possibility in monkeys. Given that monkeys rec- ognize a schematic face as a member of a facial group, pos- sibly a task using naturalistic stimuli will produce a clearer  pop-out effect. So far, however, a very limited number of  studies of visual searches using realistic faces and varied  distracters have been conducted with non-human primates.  One of the few examples is a series of chimpanzee studies  (Tomonaga and Imura 2015). In that series, chimpanzees  detected conspecific faces more efficiently than they detected  cars and houses. The efficient detection was also shown for  adult human faces, with which the chimpanzees were rich  in visual experience. The underlying mechanisms of facial  recognition may explain the similarity in the processing of  both humans and chimpanzees.

From evidence that Old World monkeys (e.g., baboons  and macaques) demonstrated various facial recognition  abilities (Parr and Hecht 2011) and that our macaque study  (Nakata et al. 2014) showed a pop-out effect during the  search for a schematic face, it is possible that the facial rec- ognition abilities of monkeys follow those of apes. In addi- tion, it is important to more naturally examine visual search  in monkeys by allowing them to freely view the objects.  Macaque studies exploring efficiency in searching for faces  under a free viewing condition (overt search) may provide  greater understanding of the facial processing in a search  task with a natural face and natural objects.

In the present study, we examined the efficient search for  a face by Japanese macaques (Macaca fuscata) and humans  under a free viewing condition. The primary purpose of the  study was to determine the factors involved in the efficient  detection of a face among non-face objects by Japanese  macaques and humans. The study’s secondary purpose was  to explore the extent and the limitations of the facial rec- ognition efficiency of both species. In Experiment 1, we  examined the categorical effect for efficient detection by  using several kinds of natural faces (e.g., conspecific faces  and faces of other-species members). We hypothesized that  each species would exhibit more efficient searches for faces  of their own and similar species’ faces and less efficient  searches for other-species faces that are clearly different  from conspecific faces, given that the search efficiency for  faces is likely to be based on the subjects’ familiarity with  conspecific faces and/or the social relevance of conspecific  faces. In Experiment 2, we further examined which facial  information is important for the pop-out effect by modulat- ing the faces’ spatial frequency components.



## Methods

## Animals

Two Japanese monkeys (M. fuscata; one male, one female;  ages 4–5 years; weight  $4.0{-}7.0\,\mathrm{kg})$  were used for the experi- ments. They were housed alone, one per cage, but allowed  visual and auditory communication with Japanese monkeys  in other cages. Their few human caretakers wore facemasks  and protective clothing to limit the animal’s experience of  whole human faces.

## Human participants

The study’s human participants were 8 university students (2  females, 6 males; ages 19–25 years with an average of 19.8;  all Japanese) having normal or corrected-to-normal vision.

## Stimuli

Target stimuli were 35 colored or gray-scale facial images  chosen from the PrimFace database (http://visiome.neuroinf. jp/primface) or our own face set. One set of 28 faces was  always presented without filtering. The remaining 7 faces  were presented with spatial frequency filtering. A wide  variety of non-face objects from our own stimulus set (480  photographs) were also included as distracter stimuli, after  standardization of their dimensions (for example, see Fig. 1).  Some of the non-face objects were human-made, namely  buildings, clothing, electric appliances, furniture, kitchen  goods, miscellaneous goods, stationery goods, tools, vehi- cles, and food products. Others were natural objects, namely  trees, flowers, rocks, vegetables, fruits, and natural scenes  (Fig. S1 and Table S1). The images were adjusted to the  same standard size ( $.70\times70$ pixels, approximately  $2.0^{\circ}$  of  visual angle). The image processing was carried out with the  use of Adobe Photoshop CS2.

## Apparatus

The monkeys were head-restrained and seated in a primate  chair during the experiment. To record eye movements of  monkeys, a scleral search coil was implanted in one eye. A 

and a Japanese monkey face, respectively, used in Experiment 1. (c,  d) A high-pass filtered human face for human participants and a low- pass filtered monkey face for monkey subjects used in Experiment 2.  Photographs of human faces and objects were prepared by R. Nakata.  Photographs of monkey faces by PrimFace database: http://visiome. neuroinf.jp/primface real-time experimental control system, TEMPO (Reflective  Computing, USA), run on a dedicated personal computer  was used to generate the stimuli, control the behavioral task  events, and record eye movements with a time resolution  of  $4~\mathrm{ms}$ . For humans, we used a keypad to register their  responses. E-Prime® (Psychology Software Tools, Inc., UK)  generated the stimuli, controlled behavioral task events and  recorded data.



## Procedure

### Behavioral task

In both experiments, the subjects searched a face (the target  stimulus) among non-face objects (distracter stimuli). Dur- ing the experiments, subjects could freely view the stimuli  (overt search).

For monkeys, the trials began with the presentation of a  circle at the center of the CRT monitor. After the monkey  fixed its eyes on the circle for an average of  $2000\,\mathrm{ms}$  (range:   $1500–2500\,\mathrm{ms}\mathrm{.}$ ), both target and distracters were simultane- ously presented. The present distracters were selected ran- domly from all distracter stimuli, regardless of the category.  The screen position of each stimulus was selected randomly  from 20 possible positions. If the monkey fixed its eyes on  the target for more than  $1000\,\mathrm{ms}$  (a correct trial), a reinforce- ment ( $0.2\;\mathrm{ml}$  of apple juice) was delivered. Reaction times  were defined as the interval between onset of the search  stimulus array and the starting point of correct fixation on  the target stimuli. After delivery of the reward, there was an  intertrial interval (ITI) of approximately  $10~\mathrm{s}$ on average.  However, if the monkey fixed its eyes on any one of the dis- tracter stimuli for more than  $1000~\mathrm{ms}$  (an error trial), then  the screen blackened, and the ITI began. Error trials were  not rewarded.



The humans’ behavioral task sequence was identical to  that of the monkeys’ in terms of the presentation of the fixa- tion point and search stimulus array, but for the behavioral  response the human participants were required to use a key- pad to indicate when and where the target was presented.  They were required to respond by pressing any key as soon  as possible when they found the target. After that, four num- bered squares were presented on the display. The screen  positions of the four squares corresponded to each of the  four quadrants: upper left, upper right, lower left, and lower  right. The participants were required to identify where the  target was on the four quadrants by pressing the appropriate  key on a keyboard. Accordingly, reaction times were defined  as the interval between onsets of the search stimulus array  and first keypad response.

### Experimental conditions

In Experiment 1, we used 4 types of target stimuli, 7 stimuli  per type, which corresponded to the following 4 conditions:  (1) a frontal view of the face of a member of the subject’s  species or race (Japanese monkey faces for Japanese mon- key subjects and Japanese faces for human participants, all  of whom were Japanese), (2) a back-of-the head view of a  member of the subject’s species (backs of the head), (3) a  front face of a member of the subject’s closely related spe- cies or race (rhesus monkey faces for Japanese monkey sub- jects and Caucasian faces for human participants), and (4) a  front face of a member of a species different from the sub- ject’s species (human faces, Japanese faces, and Caucasian  faces, for monkeys and Macaque faces, Japanese monkey  faces, and rhesus monkey faces for humans).



In Experiment 2, we used 7 frontal face images of dif- ferent individuals from the subject’s own species for target  stimuli. We set 4 experimental conditions, modulating spa- tial frequency content in each face using either of two types  of high-pass cutoff filters,  $(1)>16.8$  cycles/face width ( $\mathrm{c}/\mathrm{\Omega}$  fw) (HPF-16.8 faces) or  $(2)>12.6$ c/fw, or either of two  types of low-pass cutoffs,  $(3)<8.3$  c/fw (LPF-8.3 faces) or   $(4)<4.8$  c/fw. Average stimulus luminance did not differ  according to image type.

### Experimental blocks and sessions

For monkeys, the present study consisted of 2 consecutive  blocks corresponding to the Experiments 1–2. In each block,  we first trained the monkey to search for a target among 3  distracters under 4 different conditions. (The conditions are  described above under (1)–(4) in Experimental conditions.)  In each condition, we trained the monkeys using 3 target  stimuli with one target per trial. The monkeys performed  480 trials per session (4 conditions  $\times3$  target stimuli  $\times\;20$   possible positions on the screen $\times~2$  repetitions) with 1 or  2 sessions per day. This training phase continued until the  monkey achieved  $99\%$  correct performance in 2 consecu- tive sessions. If the monkey achieved this criterion, we next  introduced a test phase: in addition to the same number (i.e.,  3) of distracters as in the training phase, 7, 11, and 19 dis- tracters were used to test the effect of different numbers of  distracters (array sizes of 8, 12, and 20). All methods in  the test phase duplicated the training phase just before the  test phase except for target stimuli, i.e., some of the targets  were new, although they were the same type as those used in  the training phase. Therefore, a block used 28 target stimuli  (12 of the stimuli were used in both the training and test  phases, and 16 were newly used only in the test phases). A  test session consisted of 528 trials (480 duplicates of the  training phase and 48 test trials: 3 array sizes  $\times4$  differ- ent conditions  $\times4$  target stimuli). The test phase continued  for 8 sessions, regardless of the behavioral results for the  particular animal. After the 8th session of the test phase  in the first block (Experiment 1), the initial session in the  training phase of the next block (Experiment 2) began. In  the test trials, the positions of the target faces were limited  to 8 positions selected from 20. Across the 8 sessions of  the test phase, each target appeared once at each position.  Therefore, the average eccentricities of each target in the test  trials were equalized.

Humans participated only in the test phases. There were  384 trials: 240 with an array size of 4 and 144 with the  remaining array sizes. They participated Experiment 1 and  Experiment 2 consecutively in 1 day with a short break  between the two experiments.

## Data analysis

We collected data on the reaction times in each subject  and calculated the means for each condition. We analyzed  the data by conducting an analysis of variance (ANOVA).  For the macaques, we analyzed the data using a two-way  ANOVA with the factors of the array size, i.e., the number  of stimuli (target plus distracters: 8, 12, 20), and the varia- tions in the target object (the four conditions in each experi- ment) by using each session data of the two macaques (eight  sessions each) as statistical subjects. The variations across  sessions were serving to estimate error variance. For the  humans, a two-way ANOVA was used with the factors of the  array size and the variations in the target object. Previous  studies have used the slope of reaction times to search for a  target among an increasing number of distracter items as a  measure of search efficiency (Wolfe 2001), and we therefore  further analyzed the slopes of the mean reaction time as a  function of array size (ms/item) for each data set, using a  linear regression.

## Experiment 1

## Results

### The macaques

The two monkeys required 17 and 30 sessions, respec- tively, to satisfy the  $99\%$  criterion in the training phase. The  mean correct-response rate of the test phase was  $99.48\%$   ( $\mathrm{SE}=0.19\}$ , and there were no significant differences in  the mean correct-response rate between the 4 experimental  conditions. There were also no significant differences in the  mean reaction times between the conditions in the array size  of 4 in this phase (Fig. S2).

We compared the mean reaction times for the three  increased array sizes using new targets in the test phase.  Figure 2a shows the mean reaction times for these condi- tions. The ANOVA revealed significant main effects of the  array size  $(F_{2,30}\,{=}\,10.43,P<0.001$  $P<0.001$ ,  $\eta^{2}\,{=}\,0.06)$  and the vari- ations in target objects ( $(F_{3,45}=3.25$ ,  $P<0.05$ ,  $\eta^{2}=0.05)$ .  Notably, a significant interaction between these two factors   $\eta^{2}=0.07)$  $(F_{6,90}=2.70,P<0.05,\eta^{2}=0.07)$  was also revealed. With  respect to the array size, a simple main effect was confirmed  for human faces  $(F_{2,30}=10.89$ ,  $P<0.001$ ,  $\eta^{2}=0.66)$  and  backs of the head  $(F_{2,30}=5.37$ ,  $P<0.01$ ,  $\eta^{2}=0.33)$ , but  not for the faces of the two species of monkey (Japanese  macaque and rhesus macaque).



The analysis of the slopes of the mean reaction time as a  function of the array size (Fig. 2a) showed flatter gradients  for the Japanese macaque faces ( $(-~1.68~\mathrm{ms}$  per item) and  rhesus macaque faces  $(-~0.09~\mathrm{ms}$  per item) compared to  those for the backs of the head (38.12 ms per item) and the  human faces  $\mathrm{\Delta}54.66\:\mathrm{ms}$  per item).

### The humans

The mean correct-response rate in the test phase was  $99.71\%$    $(\mathrm{SE}=0.14)$ , and there were no significant differences in this  rate between the 4 experimental conditions. There were also  no significant differences in the mean reaction times between  the conditions with the array size of 4 in this phase (Fig. S1).

Figure 2b shows the mean reaction times for the three  increased array sizes. The ANOVA revealed significant main  effects of the array size  $(F_{2,14}=8.97$ ,  $P<0.01$ ,  $\eta^{2}=0.09)$   and the variations in target objects  $(F_{3,21}\,=\,34.291,$ ,  ,  $P<0.001$  $\eta^{2}=0.43)$ , and a significant interaction between  these two factors  $(F_{6,42}=2.67$ ,  $P<0.05$ ,  $\eta^{2}=0.08)$ . With  respect to the array size, a simple main effect was confirmed  for Macaque faces  $(F_{2,30}=7.84\$ ,  $P<0.001$ ,  $\eta^{2}=0.45)$  and  backs of the head  $(F_{2,30}=9.24.$ ,  $P<0.001$ ,  $\eta^{2}=0.53)$ , but  not for the human faces (both Japanese and Caucasian faces).

The analysis of the slopes of the mean reaction time as a  function of the array size (Fig. 2b) showed flatter gradients  for the Japanese faces  $(-~0.61$  ms per item) and Caucasian  faces ( $7.54\,\mathrm{ms}$  per item) compared to those for the backs of  the head  $(43.70\,\mathrm{ms}$  per item) and Macaque faces  $(39.82\:\mathrm{ms}$  per item).

## Discussion

We tested the Japanese macaques’ and human participants’  face searching ability by administering similar tasks. The  realistic faces were searched efficiently among varied dis- tracters by monkeys. These results replicated the previous  finding (Nakata et al. 2014) in which monkeys searched for  a face-like drawing among non-varied distracters, by using  more natural settings. No significant increases in the reac- tion times for the increase in the array sizes were observed  when the frontal face of own genus for the monkeys was  searched by the Japanese macaques and the humans. It thus  seems reasonable that not only humans and chimpanzees  but also Japanese macaques detect the frontal-view faces  of conspecifics efficiently. The range of distracter objects  presented in this experiment was exceedingly large, and the  monkeys’ visual experience of some of these distracters  might be different from that of the human participants. In  other words, the monkeys might not have encountered most  of the human-made objects in their lives, whereas human  participants presumably had sufficient experience with all  presented objects. Nevertheless, the pattern of results was  almost the same for humans and monkeys. Therefore, the  difference in daily experiences for some of the distracter  objects did not appear to affect the present results.



The faces used here, i.e., both the two types of human  faces (Japanese and Caucasian) and the two types of  Macaque faces (Japanese macaque and rhesus macaque),  share several facial features in terms of function and shape,  and the arrangement of the facial features is largely com- mon. These similarities between human faces and Macaque  faces — in terms of sensitivity to the first-order spatial con- figuration among facial features (Diamond and Carey 1986)  as “a face” — were less effective for the efficient detection  of the faces. In contrast, the information from faces that is  used to infer “a species” was effective for the detection. The  macaques in this experiment detected the macaque faces  more efficiently than they detected the human faces, and the  human participants detected human faces more efficiently  than the macaque faces. These results concur with the results  of Experiment 4 in the above-mentioned study of chimpan- zees (Tomonaga and Imura 2015) in which the chimpanzees  did not efficiently detect faces of Japanese macaques.

The advantage in the recognition of congeneric faces for  monkeys as well as conspecific faces for humans has been  referred to as own- or other-species effect (OSE) in behav- ioral studies (Campbell et al. 1997; Pascalis and Bachevalier  1998; Dufour et al. 2006; Scott and Fava 2013) and neurosci- ence studies (de Haan et al. 2002; Scott et al. 2005; Sigala  et al. 2011). Our present findings suggest a robust OSE on  the efficient detection of faces. We observed that the efficient  detection of a face is likely to occur not toward every face  stimulus, which shared the first-order spatial configuration,  but toward congeneric face stimuli which may be at a basic  and entry level classification (Tarr and Cheng 2003) of the  description of faces.

Numerous studies indicate that faces from a human sub- ject’s own race are recognized or discriminated better than  faces of humans of other races, known as the other-race  effect (ORE) in humans (Hugenberg et al. 2010; Anzures  et al. 2013; O’Toole and Natu 2013), which is similar to  the OSE. Japanese macaques clearly discriminated pictures  of their own species from those of closely related species  (Fujita 1987), indicating that Japanese macaques scanned  other macaques in a manner that is similar to that observed  in humans who are subject to the ORE.

In the present study, in contrast to the ORE confirmed in  previous studies, the human subjects (who were Japanese)  were able to efficiently detect the faces of another race (Cau- casian) among non-face objects. It is likely that the human  participants had experienced seeing many Caucasian faces  in real life or through the internet, although Japanese have  less experience with Caucasian faces in general. It must be  noted that Japanese faces were detected just slightly faster  than Caucasian faces. This may indicate that the Japanese  participants processed Caucasian faces as having very subtle  differences in the configuration of facial features compared  to faces of their own ethnicity. The Japanese macaques  were also observed to efficiently detect the faces of another  (macaque) species, the rhesus faces. The present results sug- gest that, contrary to the OSE, the ORE has little impact  on the efficient detection of the faces at the basic level of  classification. Thus, subtle information that is useful for the  discrimination of race is likely to be ignored when search- ing a face among non-face objects, and each face would be  processed similarly as a face.



Stronger holistic or configural processing underlies the  own-race advantage in face memory, in humans (Tanaka  et al. 2004; Michel et al. 2006) and in chimpanzees (Tomon- aga and Imura 2015). In addition, studies with human sub- jects obtained convincing evidence of the importance of  high-level, holistic face characteristics (Maurer et al. 2002;  Burke and Sulikowski 2013) for the efficient detection of  conspecific faces (Hershler and Hochstein 2005). Consid- ering that Old World monkeys use a holistic or configural  process when they recognize the differences between con- specific faces (Martin-Malivel et al. 2006; Adachi et al.  2009; Dahl et al. 2009; Gothard et al. 2009), it is likely that  the efficient detection confirmed in the present experiment  was due to a process based on the holistic information of  own-class faces rather than a process based on distinctive  facial parts.

However, in another human study (VanRullen 2006), the  pop-out effect similar to that observed in the above human  study (Hershler and Hochstein 2005) was confirmed, but it  was not limited to faces, was not holistic, and was due to  low-level properties of the images. Although the pop-out  effect of faces from a variety of heterogeneous distracters  in humans is now widely accepted, there is still room for a  considerable measure of disagreement about what informa- tion included in the face is important for the pop-out effect  (Hershler and Hochstein 2006). Therefore, in Experiment 2,  we further examined which facial information (holistic high- level information of faces or low-level visual information  including faces) is important for the pop-out effect.

## Experiment 2

## Results

### The macaques

In the training phase, the two macaques required five and  eight sessions, respectively, to satisfy the  $99\%$  criterion.  The mean correct-response rate in the subsequent test  phase was  $99.13\%$  ( $\mathrm{SE}=0.28)$ , and there were no sig- nificant differences in the rate between the experimental  conditions. There were also no significant differences in  the mean reaction times between the conditions with the  array size of 4 in this phase (Fig. S1).



Figure 3a shows the mean reaction times for the three  increased array sizes in the test phase. The ANOVA  revealed significant main effects of the array size  ,  $(F_{2,30}=25.07\$  $P<0.001$ ,  $\eta^{2}=0.08)$  and the variation  in target objects  $(F_{3,45}=17.91\$ ,  $P<0.001$ ,  $\eta^{2}=0.14)$   and a significant interaction between these two factors  ,  $(F_{6,90}=4.52$  $P<0.001$ ,  $\eta^{2}=0.09)$ . The simple main  effect of the array size was confirmed only for the HPF- 16.8 faces  $(F_{2,30}\,{=}\,30.08,P<0.001,\eta^{2}\,{=}\,0.84)$  $\eta^{2}=0.84)$ ; there were  no significant differences between the array sizes for each  of the other three types of faces.

Our analysis of the slopes of the mean reaction time  as a function of the array size showed a steeper gradient   $(34.43~\mathrm{ms}$  per item) for the HPF-16.8 faces compared to  the two low-pass filtered faces (LPF-8.3 faces,  $5.69~\mathrm{ms}$   per item; LPF-4.8 faces,  $8.64~\mathrm{ms}$  per item). The slope for  the HPF-12.6 faces (9.99 ms per item) was between these  steep and flat gradients.

### The humans

The mean correct-response rate in the test phase was  $99.64\%$   ( $(\mathrm{SE}=0.11)$ , and there were no significant differences in this  rate between the experimental conditions. There were also  no significant differences in the mean reaction times between  the conditions with the array size of 4 in this phase.

Figure 3b shows the mean reaction times for the three  increased array sizes in the test phase. The ANOVA revealed  significant main effects of the array size  $(F_{2,14}=9.27$ ,  ,  $P<0.01$  $\eta^{2}=0.12)$  and the variation in target objects   $\eta^{2}=0.34)$  $(F_{3,21}=18.80,P<0.001,\eta^{2}=0.34)$ , and a significant inter- action between these two factors  $(F_{6,42}=3.12$ ,  $P<0.05$ ,  . With respect to the array size effect, the simple  $\eta^{2}=0.08)$  main effect was confirmed for two types of HPF faces (HPF- 16.8 faces, $F_{2,30}\,{=}\,14.52,P\,{<}\,0.001,\eta^{2}\,{=}\,0.70$  $\eta^{2}\,{=}\,0.70$  and HPF-12.6  faces,  $F_{2,30}=5.48$ ,  $P<0.01$ ,  $\eta^{2}=0.26)$ . There were no  significant differences between the array sizes for the two  types of LPF faces.

Our analysis of the slopes of the mean reaction time  as a function of the array size showed steeper gradients  ( $30.30~\mathrm{ms}$  per item) for the HPF-16.8 faces and the HPF- 12.6 faces (21.81 ms per item) compared to the two types  of low-pass filtered faces (LPF-8.3 faces, 6.90 ms per item;  LPF-4.8 faces, 4.91 ms per item).

## Discussion

In the present experiment, when the array size was increased,  both the humans and macaques showed almost no or only  slight increases in reaction times when searching for the  LPF faces, whereas they showed steeper slopes for the HPF  faces; in particular, the slope for the HPF-16.8 faces showed  a clearly different trend compared to that for the LPF faces.  These results suggest that the information provided by low- spatial-frequency components in faces is more important  than the information provided by high-spatial-frequency  components for efficient facial searching among non-face  objects. This may also be true when naturalistic own-class  faces are presented as the target, as in Experiment 1.

Previous studies have shown that high-spatial-frequency  components provide information about distinct facial parts  relatively well and that low-spatial-frequency components  provide holistic information (Vuilleumier et al. 2003; Gof- faux and Rossion 2006). Actually, the LPF faces appeared  blurred, while the HPF faces looked very sharp and each  facial part appeared distinctly. Therefore, in searching for  the LPF faces, it was difficult for the subject to focus on  the individual parts of the face, and this difficulty may have  augmented the process based on holistic information. In  contrast, in searching for the HPF faces, it was easier for the  subject to focus on the individual parts of the face, which  may have augmented the process based on local informa- tion from individual facial parts. Our findings thus suggest  that the process based on holistic information which is not  ascribed to individual facial parts is more critical for effi- cient face searching than the process based on local informa- tion. In this experiment, we did not use intact faces in which  the spatial frequency contents were not modulated. It is thus  debatable whether the efficient facial searching for the LPF  faces is similar to the facial searching of intact faces (e.g.,  the own-species faces in Experiment 1). Further research is  needed to address this issue.

## General discussion

Our findings have two important implications. First, the  results for the humans and Japanese macaques showed  similar trends at most points in both experiments, suggest- ing that both species possess a mechanism that allows them  to search for a face among non-face objects in an efficient  manner. Some studies claimed that unlike chimpanzees, rhe- sus monkeys did not show the Thatcher illusion (Thomp- son 1980), a measure of face processing ability in which  changes in the configuration of facial features are difficult  to detect in an inverted face (Weldon et al. 2013). There  may be potential evolutionary differences in face processing  among primates. However, other studies have systematically  shown that monkeys perceive the Thatcher illusion (Adachi  et al. 2009; Dahl et al. 2010). Furthermore, the Thatcher  illusion was shown even in squirrel monkeys, a New World  monkey that shows great differences from humans in terms  of ecology, morphology, and anatomy (Nakata and Osada  2012). Other studies also did not find the differences in face  processing among primates (Dahl et al. 2013). Therefore,  it seems reasonable that the ability to search efficiently for  faces derives from fundamental social cognition abilities  broadly shared among primate species, although there is  room for further investigation to determine whether other  species showing social behavior than humans (Hershler and  Hochstein 2005), chimpanzees (Tomonaga and Imura 2015)  and Japanese macaques can efficiently search for a realistic  face among various non-face objects; for example, among  New World monkeys whose face recognition abilities are  somewhat similar to those of humans (Weiss et al. 2001;  Neiworth et al. 2007; Nakata and Osada 2012).



Second, efficient face detection was confirmed when  the searched-for faces had lower-spatial-frequency compo- nents and when the searched-for faces shared similar vis- ual description as “own-species faces” (e.g., faces of both  Japanese and Caucasians for the human participants and  macaque faces of both Japanese and rhesus macaques for  the monkey subjects). The slope of the increase in reaction  time to detect the target among additional distracter items  was used as a measure of search efficiency in human studies,  with a slope close to zero in efficient parallel search (Wolfe  2001). Our results showed this trend relatively well in the  searching of conspecifics faces and LPF faces. These results  suggest that (1) processing based on the holistic information  of the faces is important for the efficient detection of faces;  in other words, the holistic processing of the faces is clearly  meaningful in the earlier stage of visual processing as well  as in the later stages; and (2) the processing that is developed  due to sufficient experience with conspecific faces could be  generalized over the extent of the experience.

Under our experimental settings, why did humans and  monkeys conduct searches for the same face in different  manners? For example, why did humans search for a human  face efficiently, while monkeys searched for the same face  inefficiently? Viewed in this light, the most likely explana- tion is that, in both humans and Japanese macaques, the  faces for which the subjects searched efficiently became  specialized through perceptual narrowing in early life (Pas- calis et al. 2002) and/or daily experience in later years. The  specialization may be endowed innately or it may occur at  a very early stage of life as a study indicated that newborn  monkeys can acquire the knowledge of basic structure of  own-species faces (Sugita 2009). Such specialization would  lead to an increase in useful information (i.e., holistic infor- mation), which would further augment the efficiency of the  search.

Morphological similarities among frequently encountered  faces and infrequently encountered ones (e.g., faces of sub- jects’ closely related species) make use of similar configural  processing in the faces (Dahl et al. 2014). Regarding our pre- sent results, it is thus likely that differences in the efficient  detection of the face classes and the differences between  the OSE and the ORE in efficient face detection are due to  the differences in the morphological similarity among face  classes.

In regard to the main conclusion of this study, our results  strongly suggest that conspecific faces were detected effi- ciently among non-face objects due to a process in which  the face recognition mechanism is shared not only among  humans but also among at least one species of Old World  monkey, i.e., Japanese macaques.

Acknowledgments The authors would like to thank Mr. Yoshio Dai- mon and Mr. Takashi Kitamura for their technical assistance and  care for animals. The face images of monkeys used in this study are  provided by PrimFace database: http://visiome.neuroinf.jp/primface,  funded by Grant-in-Aid for Scientific Research on Innovative Areas,  “Face Perception and Recognition” from Ministry of Education, Cul- ture, Sports, Science, and Technology (MEXT), Japan.

Funding This study was supported by JSPS KAKENHI Grant Numbers  JP26350992, JP17K01991, JP23500465 and JP26350992) and a Grant- in-Aid for Scientific Research on Innovative Areas, “Face Perception  and Recognition” (No. 20119006) from Ministry of Education, Culture,  Sports, Science and Technology (MEXT), Japan.

## Compliance with ethical standards 

Conflict of interest All authors declare that there is no conflict of in- terest.

Human and animal rights The experimental protocols for animals were  approved by The Animal Care and Use Committee of the University  of Toyama (Permit  $\#$ A2013MED-61). All experimental protocols  conformed to the National Institutes of Health (NIH) guidelines for  the humane care and use of laboratory animals and to the Association  for Research in Vision and Ophthalmology statement for use of ani- mals in ophthalmic and vision research. The experimental protocols  for human participants accorded with the Declaration of Helsinki and  were approved by the Research Ethics Committees of the University of  Toyama (Permit #s 130 and 24-41). The human participants provided  written informed consent after an explanation of the study’s purpose.

## References

Adachi I, Chou DP, Hampton RR (2009) Thatcher effect in mon- keys demonstrates conservation of face perception across pri- mates. Curr Biol 19:1270–1273. https://doi.org/10.1016/j. cub.2009.05.067 Anzures G, Quinn PC, Pascalis O, Slater AM, Tanaka JM, Lee K (2013)  Developmental origins of the other-race effect. Curr Dir Psychol  Sci 22:173–178. https://doi.org/10.1177/0963721412474459 Becker DV, Anderson US, Mortensen CR, Neufeld SL, Neel R (2011)  The face in the crowd effect unconfounded: happy faces, not angry  faces, are more efficiently detected in single- and multiple-target  visual search tasks. J Exp Psychol Gen 140:637–659. https://doi. org/10.1037/a0024060 Burke D, Sulikowski D (2013) The evolution of holistic process- ing of faces. Front Psychol 4:11. https://doi.org/10.3389/ fpsyg.2013.00011 Campbell R, Pascalis O, Coleman M, Wallace SB, Benson PJ (1997)  Are faces of different species perceived categorically by human  observers? Proc R Soc Lond B 264:1429–1434. https://doi. org/10.1098/rspb.1997.0199 Dahl CD, Wallraven C, Bülthoff HH, Logothetis NH (2009) Humans  and macaques employ similar face-processing strategies. Curr  Biol 19:509–513. https://doi.org/10.1016/j.cub.2009.01.061 Dahl CD, Logothetis NK, Bülthoff HH, Wallraven C (2010) The  Thatcher illusion in humans and monkeys. Proc Biol Sci  277:2973–2981. https://doi.org/10.1098/rspb.2010.0438 Dahl CD, Rasch MJ, Tomonaga M, Adachi I (2013) Development pro- cesses in face perception. Sci Rep 3:1044. https://doi.org/10.1038/ srep01044 Dahl CD, Rasch MJ, Chen CC (2014) The other-race and other-species  effects in face perception—a subordinate-level analysis. Front  Psychol 5:1068. https://doi.org/10.3389/fpsyg.2014.01068 de Haan M, Pascalis O, Johnson MH (2002) Specializa- tion of neural mechanisms underlying face recognition in  human infants. J Cogn Neurosci 14:199–209. https://doi. org/10.1162/089892902317236849 Diamond R, Carey S (1986) Why faces are and are not special: an  effect of expertise. J Exp Psychol Gen 115:107–117. https://doi. org/10.1162/089892902317236849 Dufour V, Pascalis O, Petit O (2006) Face processing limitations to  own species in primates: a comparative study in brown capuchins,  Tonkean macaques and humans. Behav Process 73:107–113.  https://doi.org/10.1016/j.beproc.2006.04.006 Fujita K (1987) Species recognition by five macaque monkeys. Pri- mates 28:353–366. https://doi.org/10.1007/BF02381018 Goffaux V, Rossion B (2006) Face are “spatial”–holistic face  perception is supported by low spatial frequencies. J Exp  Psychol Hum Percept Perform 32:1023–1039. https://doi. org/10.1037/0096-1523.32.4.1023 Gothard KM, Erickson CA, Amaral DG (2009) How do rhesus  monkeys (Macaca mulatta) scan faces in a visual paired com- parison task? Anim Cogn 7:25–36. https://doi.org/10.1007/ s10071-003-0179-6 Hansen C, Hansen R (1988) Finding the face in the crowd: an anger  superiority effect. J Pers Soc Psychol 54:917–924. https://doi. org/10.1037/0022-3514.54.6.917 Hershler O, Hochstein S (2005) At first sight: a high-level pop out effect  for faces. Vision Res 45:1707–1724. https://doi.org/10.1016/j. visres.2004.12.021 Hershler O, Hochstein S (2006) With a careful look: still no low-level  confound to face pop-out. Vision Res 46:3028–3035. https://doi. org/10.1016/j.visres.2006.03.023 Hugenberg K, Young SG, Bernstein MJ, Sacco DF (2010) The cat- egorization-individualization model: an integrative account of  the other race recognition deficit. Psychol Rev 117:1168–1187.  https://doi.org/10.1037/a0020463 Levin DT (1996) Classifying faces by race: the structure of face cat- egories. J Exp Psychol Learn Mem Cogn 22:1364–1382. https:// doi.org/10.1037/0278-7393.22.6.1364 Martin-Malivel J, Mangini MC, Fagot J, Biederman I (2006) Do  humans and baboons use the same information when categoriz- ing human and baboon faces? Psychol Sci 17:599–607. https:// doi.org/10.1111/j.1467-9280.2006.01751.x Maurer D, Le Grand R, Mondloch CJ (2002) The many faces of  configural processing. Trends Cogn Sci 6:255–260. https://doi. org/10.1016/S1364-6613(02)01903-4 Michel C, Caldara R, Rossion B (2006) Same-race faces are perceived  more holistically than other-race faces. Vis Cogn 14:55–73.  https://doi.org/10.1111/j.1467-9280.2006.01752.x Montoute T, Tiberghein G (2001) Unconscious familiarity and local  context effects on low-level face processing: a reconstruction  hypothesis. Conscious Cogn 10:503–523. https://doi.org/10.1006/ ccog.2001.0524 Nakata R, Osada Y (2012) The Thatcher illusion in squirrel mon- keys (Saimiri sciureus). Anim Cogn 15:517–523. https://doi. org/10.1007/s10071-012-0479-9 Nakata R, Eifuku S, Tamura R (2014) Effects of tilted orientations and  face-like configurations on visual search asymmetry in macaques.  Anim Cogn 17:67–76. https://doi.org/10.1007/s10071-013-0638-7 Neiworth JJ, Hassett JM, Sylvester CJ (2007) Face processing in  humans and new world monkeys: the influence of experimen- tal and ecological factors. Anim Cogn 10:125–134. https://doi. org/10.1007/s10071-006-0045-4 O’Toole AJ, Natu V (2013) Computational perspectives on the other- race effect. Vis Cogn 21:1121–1137. https://doi.org/10.1080/13 506285.2013.803505 Parr LA, Hecht EE (2011) Face perception in non-human primates.  In: Calder AJ, Rhodes G, Johnson MH, Haxby JV (eds) The  oxford handbook of face perception, 1st edn. Oxford University  Press, New York, pp 691–706. https://doi.org/10.1093/oxfor dhb/9780199559053.013.0035 Pascalis O, Bachevalier J (1998) Face recognition in primates: a cross- species study. Behav Process 43:87–96. https://doi.org/10.1016/ S0376-6357(97)00090-9 Pascalis O, de Haan M, Nelson CA (2002) Is face processing species- specific during the first year of life? Science 296:1321–1323.  https://doi.org/10.1126/science.1070223 Scott LS, Fava E (2013) The own-species face bias: a review of devel- opmental and comparative data. Vis Cogn 21:1364–1391. https:// doi.org/10.1080/13506285.2013.821431 Scott LS, Shannon RW, Nelson CA (2005) Behavioral and electro- physiological evidence of species-specific face processing. Cogn  Affect Behav Neurosci 5:405–416. https://doi.org/10.3758/ CABN.5.4.405 Sigala R, Logothetis NK, Rainer G (2011) Own-species bias in the  representations of monkey and human face categories in the  primate temporal lobe. J Neurophysiol 105:2740–2752. https:// doi.org/10.1152/jn.00882.2010 Sugita Y (2009) Innate face processing. Curr Opin Neurobiol 19:39– 44. https://doi.org/10.1016/j.conb.2009.03.001 Tanaka JW, Kiefer M, Bukach CM (2004) A holistic account of the  own-race effect in face recognition: evidence from a cross- cultural study. Cognition 93:B1–B9. https://doi.org/10.1016/j. cognition.2003.09.011 Tarr MJ, Cheng YD (2003) Learning to see faces and objects.  Trends Cogn Sci 7:23–30. https://doi.org/10.1016/ S1364-6613(02)00010-4 Thompson P (1980) Margaret Thatcher: a new illusion. Perception  9:483–484. https://doi.org/10.1068/p090483 Tomonaga M, Imura T (2015) Efficient search for a face by chimpan- zees (Pan troglodytes). Sci Rep 5:11437. https://doi.org/10.1038/ srep11437 Tong F, Nakayama K (1999) Robust representations for faces: evi- dence from visual search. J Exp Psychol Hum Percept Perform  25:1016–1035. https://doi.org/10.1037//0096-1523.25.4.1016 Treisman AM, Gelade G (1980) A feature-integration the- ory of attention. Cogn Psychol 12:97–136. https://doi. org/10.1016/0010-0285(80)90005-5 VanRullen R (2006) On second glance: still no high-level pop-out effect  for faces. Vision Res 46:3017–3027. https://doi.org/10.1016/j. visres.2005.07.009 Vuilleumier P, Armony JL, Driver J, Dolan R (2003) Distinct spatial  frequency sensitivities for processing faces and emotional expres- sions. Nat Neurosci 6:624–631. https://doi.org/10.1038/nn1057 Weiss DJ, Kralik JD, Hauser MD (2001) Face processing in cotton- top  tamarins (Saguinus oedipus). Anim Cogn 3:191–205. https://doi. org/10.1007/s100710000076 Weldon KB, Taubert J, Smith CL, Parr LA (2013) How the Thatcher  illusion reveals evolutionary differences in the face processing  of primates. Anim Cogn 16:691–700. https://doi.org/10.1007/ s10071-013-0604-4 Wolfe JM (2001) Asymmetries in visual search: an introduction.  Percept Psychophys 63:381–389. https://doi.org/10.3758/ BF03194406

                                              

                                             

                                            

                                           

                                          

                                         

                                        

                                       

                                      

                                     

                                    

                                   

                                  

                                 

                                

                               

                              

                             

                            

                           

                          

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 


Badyaev AV, Young RL, Hill GE, Duckworth RA (2008) Evolution of sex-biased maternal effects in birds. IV. Intra-ovarian growth dynamics can link sex-determination and sex-specific acquisition of resources. Journal of Evolutionary Biology, 21, 449–460.

Brush AH, Power DM (1976) House finch pigmentation: carotenoid metabolism and the effect of diet. Auk, 93, 725–739.

Dhondt AA, Badyaev AV, Dobson AP et al. (2006) Mycoplasmal conjunctivitis spreads more slowly in native than in introduced range of the host. Ecohealth, 3, 95–102.

Duckworth RA, Badyaev AV, Farmer KL, Hill GE, Roberts SC (2003a) First case of Mycoplasma gallisepticum infection in the western range of the House Finch (Carpodacus mexicanus). Auk, 120, 528–530.

Duckworth RA, Badyaev AV, Parlow AF (2003b) Elaborately ornamented males avoid costly parental care in the house finch (Carpodacus mexicanus): a proximate perspective. Behavioral Ecology and Sociobiology, 55, 176–183.

Gautschi B, Tenzer I, Müller JP, Schmid B (2000a) Isolation and characterization of microsatellite loci in the bearded vulture (Gypaetus barbatus) and cross-amplification in three Old World vulture species. Molecular Ecology, 9, 2193–2195.

Gautschi B, Widmer A, Koella J (2000b) Isolation and characteriza- tion of microsatellite loci in the dice snake (Natrix tessellata). Molecular Ecology, 9, 2191–2193.

Hawley DM, Hanley D, Dhondt AA, Lovette IJ (2006) Molecular evidence for a founder effect in invasive house finch (Carpodacus mexicanus) populations experiencing an emergent disease epidemic. Molecular Ecology, 15, 263–275.

Hill GE (2002) A Red Bird in a Brown Bag: The Function and Evolution of Colorful Plumage in the House Finch. Oxford University Press, Oxford, UK.

Jamieson A, Taylor SCS (1997) Comparisons of three probability formulae for parentage exclusion. Animal Genetics, 28, 397–400.

Lindstedt E, Oh KP, Badyaev AV (2006) Ecological, social, and genetic contingency of extrapair behavior in a socially monog- mous bird. Journal of Avian Biology, 38, 214–238.

Oh KP, Badyaev AV (2006) Adaptive genetic complementarity in mate choice coexists with preference for elaborate sexual traits. Proceedings of the Royal Society B: Biological Sciences, 273, 1913–1919. Oh KP, Badyaev AV (2008) Evolution of adaptation and mate choice: parental relatedness affects expression of phenotypic vari- ance in a natural population. Evolutionary Biology, 35, 111–124.

Pritchard JK, Stephens M, Donnelly P (2000) Inference of population structure using multilocus genotype data. Genetics, 155, 945–959. Queller DC, Strassmann JE, Hughes CR (1993) Microsatellites and kinship. Trends in Ecology & Evolution, 8, 285–288.

Raymond M, Rousset F (1995) GenePop (version 1.2): population genetics software for exact tests and ecumenicism. Journal of Heredity, 86, 248–249.

Tenzer I, degli Ivanissevich S, Morgante M, Gessler C (1999) Identification of microsatellite markers and their application to population genetics of Venturia inaequalis. Phytopathology, 89, 748–753.

Veit RR, Lewis MA (1996) Dispersal, population growth, and the Allee effect: dynamics of the house finch invasion of eastern North America. American Naturalist, 148, 255–274.

Wang Z, Baker AJ, Hill GE, Edwards SV (2003) Reconciling actual and inferred population histories in the house finch (Carpodacus mexicanus) by AFLP analysis. Evolution, 57, 2852–2864.

Wootton JT (1987) Interspecific competition between introduced house finch populations and two associated passerine species. Oecologia, 71, 325–331.

doi:  10.1111/j.1755-0998.2009.02555.x

 $@$  2009 Blackwell Publishing Ltd

# Isolation and characterization of new microsatellite markers  for rose bitterlings, Rhodeus ocellatus

Y. SHIRAI,\* S. IKEDA† and S. TAJIMA‡ \*Kagawa Prefectural Research Institute for Environmental Sciences and Public Health, 5-3-105 Asahimachi, Takamatsu, 760-0065,  Japan, †Gene Research Center, Kagawa University, 2393, Ikenobe, Miki-cho, Kita-gun, Kagawa 761-0795, Japan, ‡Department of Life  Sciences, Kagawa University, 2393, Ikenobe, Miki-cho, Kita-gun, Kagawa 761-0795, Japan

## Abstract

The Japanese rose bitterling (Rhodeus ocellatus kurumeus because of hybridization and competition from an invasive alien subspecies (Rhodeus ocellatus ocellatus dinucleotide repeat specific polymerase chain reaction. The number of alleles per locus and R. o. kurumeus were lower than those in R. o. ocellatus. microsatellite markers were successfully cross-amplified in three Acheilognathinae species.

Keywords: Acheilognathinae, cross-species amplification, microsatellite markers, Rhodeus ocellatus, rose bitterling Received 24 April 2008; revision accepted 10 December 2008 Correspondence: Y. Shirai, Fax: +81-87-825-0408; E-mail: vg7552@pref.kagawa.lg.jp Two subspecies of the rose bitterling, Rhodeus ocellatus kurumeus (Teleostei:Cyprinidae) and R. o. ocellatus, inhabit Japan. Rhodeus o. kurumeus is endemic and facing extinction brought about by both hybridization and competition from an invasive alien subspecies, R. o. ocellatus. Rhodeus o. kurumeus is currently listed as a Critically Endangered species in the Red Data Book of the Japanese Ministry of the Environment. Clear and convenient identification of the two subspecies is needed to protect R. o. kurumeus from extinction. However, there have been no adequate molecular markers for R. ocellatus. We present the characterization of 11 new microsatellite markers for R. ocellatus and their applicability to three related species.

  

 



We amplified DNA fragments from the genomic DNA sample of R. o. kurumeus using dinucleotide repeat specific polymerase chain reaction (PCR) as described by Fisher et al. (1996). The PCR mixture consisted of  $20\,\mathrm{ng}$  genomic DNA, -anchored primer  $\scriptstyle({\mathrm{PCT}}_{6};$  $100\,\mathrm{nmol}\,5^{\prime}.$  KKVRVRVCTCTCTC TCTCT, or  $\mathrm{PCA}_{6};$  KKBRBRBCACACACACACA),  $800\,\upmu\mathbf{M}$  dNTPs,  $0.02\ \mathrm{U}/\upmu\mathrm{L}$  Taq DNA Polymerase (NEB), and  $1\times$  ThermoPol Buffer [NEB, 20 mm Tris-HCl,  $10\,\mathrm{{mm}\,(N H_{4})_{2}S O_{4}},$   Triton $.{\mathrm{X}}{\mathrm{-}}100,{\mathrm{pH}}\,8.8,$ ) in $10–\upmu\mathrm{L}$  $10\,\mathrm{{mm\,KCl}},2\,\mathrm{{mm\,MgSO}_{4}},0.1\%$  reaction volume. The temperature-cycle conditions were as follows: initial denaturation for 3 min at $94\,^{\circ}C;$  followed by 40 cycles of $25\,\mathrm{s}$  at $94\ {}^{\circ}\mathrm{C},60\,\mathrm{s}$  at $57\,^{\circ}C,$  and 45 s at $72\,^{\circ}C;$ ; and final extension for 7 min at  $72\,^{\circ}\mathrm{C}$ . The amplified DNA was cloned into pGEM-T easy (Promega) using TA cloning pro- tocol and the ligations were chemically transformed into DH5α (Toyobo). The insert fragments with enough length to design PCR primers were sequenced on a PRISM 3100 Automated Capillary DNA Sequencer (Applied Biosystems) using a DYEnamic ET Terminator Cycle Sequencing Kit (GE Healthcare Bio-Sciences). Eleven primer pairs were designed using Primer 3 software (Rozen & Skaletsky 2000) from the sequence data of 70 clones. 



To evaluate the applicability of the microsatellite markers, we used the genomic DNA samples prepared from 20 individuals each of both subspecies. Individuals of R. o. kurumeus and R. o. ocellatus were collected from two ponds in the same area (10 each) and one stream, respectively. The PCR mixture contained $20\,\mathrm{ng}$  of genomic DNA, $100\,\mathrm{nM}$  each of forward and reverse primer, $800\,\upmu\mathbf{M}$  dNTPs, $1.3\,\upmu\mathbf{M}$  FITC- dUTP (Roche Diagnostics),  $0.02\,\mathrm{U/mL}$  of Taq DNA Poly- merase (NEB), and  $1\times$ ThermoPol Buffer in  $10–\upmu\mathrm{L}$  reaction volume. The temperature-cycle conditions were as follows: initial denaturation for 3 min at $94\,^{\circ}C,$ ; followed by 30 cycles —, no amplification.



of 15 s at  $94\,^{\circ}C,$   $30\,\mathrm{s}$  at  $56\,^{\circ}{\cal C},$  and  $20\,\mathrm{s}$ at  $72\,^{\circ}{\cal C};$  and final extension for  $7\,\mathrm{min}$  at  $72\,^{\circ}\mathrm{C}$ . The amplified microsatellite markers were mixed with three volumes of formamide dye solution  $(95\%$  formamide,  $10\mathrm{\mm\EDTANa_{2}},\,10\mathrm{\mg/mL}$  bluedextran), heat denatured at  $94\,^{\circ}\mathrm{C}$  for  $3\,\mathrm{min}$  and run with MapMarker (BioVenture) on a  $5\%$  PAGE-PLUS (Amresco) — 8 M urea denaturing gel using a DSQ-1000 Slab Gel DNA Sequencer (Shimadzu Biotech). 

The characteristics of newly developed microsatellite markers are shown in Table 1. Estimations of the expected and observed heterozygosities and the assessment of Hardy– Weinberg equilibrium and linkage disequilibria were carried out using GenePop version 3.4 (Raymond & Rousset 1995). In R. o. kurumeus, five and six out of 11 microsatellite markers were monomorphic and polymorphic, respectively. Exact tests of deviation from Hardy–Weinberg equilibrium and estimates of linkage disequilibrium in the case of poly- morphic markers were not significant after sequential Bonferroni correction (Rice 1989). Conversely, in R. o. ocellatus, all the microsatellite markers were multi-allelic. The hetero- zygosities of all markers except RC364 were high, with numbers of alleles ranging from six to 12. Exact tests of deviation from Hardy–Weinberg equilibrium were significant in RC306, RC364 and RC688 after sequential Bonferroni correction. The deviation may result from the small sample size or the presence of null alleles. The contribution of the latter was analysed with Micro-Checker (van Oosterhout et al. 2004). The estimated frequencies of null alleles at RC306, RC364 and RC688 were 0.16, 0.38 and 0.14, respectively, indicating that the deviation is likely to be caused by null alleles. The linkage disequilibria in R. o. ocellatus were not significant in all locus pairs after sequential Bonferroni correction.



Twenty individuals of R. o. kurumeus were classified into 18 genotypes using a combination of six markers  $({\mathrm{RC}}236,$  292, 363, 364, 625 and 772). In addition, all the microsatellite markers except RC236 were cross-amplified successfully in three other Acheilognathinae species (Table 2). Therefore, these microsatellite markers are not only useful for research of the genetic structure of R. o. ocellatus and the conservation of R. o. kurumeus, but are also possibly applicable to other related endangered Acheilognathinae species such as Tanakia tanago.

## Acknowledgements

We would like to thank Y. Kanoh for the R. o. kurumeus sample and H. Kubota for the R. o. ocellatus, T. lanceolata, A. melanogaste and A. tabira samples.

## References

Fisher PJ, Gardner RC, Richardson TE (1996) Single locus micro- satellites isolated using $5^{\prime}$  anchored PCR. Nucleic Acids Research, 24, 4369–4371.

van Oosterhout C, Hutchinson WF, Wills DPM, Shipley P (2004) Micro-Checker: software for identifying and correcting geno- typing errors in microsatellite data. Molecular Ecology Notes, 4, 535–538.

Raymond M, Rousset F (1995) GenePop (version 1.2): a population genetics software for exact test and ecumenicism. Journal of Heredity, 86, 248–249.

Rice WR (1989) Analyzing tables of statistical tests. Evolution, 43, 223–225.

Rozen S, Skaletsky H (2000) Primer 3 on the WWW for general users and biologist programmers. In: Bioinformatics Methods and Protocols: Methods in Molecular Biology (eds Krawetz S, Misener S), pp. 365–386. Humana Press, Totowa, New Jersey.
Badyaev AV, Young RL, Hill GE, Duckworth RA (2008) Evolution of sex-biased maternal effects in birds. IV. Intra-ovarian growth dynamics can link sex-determination and sex-specific acquisition of resources. Journal of Evolutionary Biology, 21, 449–460.

Brush AH, Power DM (1976) House finch pigmentation: carotenoid metabolism and the effect of diet. Auk, 93, 725–739.

Dhondt AA, Badyaev AV, Dobson AP et al. (2006) Mycoplasmal conjunctivitis spreads more slowly in native than in introduced range of the host. Ecohealth, 3, 95–102.

Duckworth RA, Badyaev AV, Farmer KL, Hill GE, Roberts SC (2003a) First case of Mycoplasma gallisepticum infection in the western range of the House Finch (Carpodacus mexicanus). Auk, 120, 528–530.

Duckworth RA, Badyaev AV, Parlow AF (2003b) Elaborately ornamented males avoid costly parental care in the house finch (Carpodacus mexicanus): a proximate perspective. Behavioral Ecology and Sociobiology, 55, 176–183.

Gautschi B, Tenzer I, Müller JP, Schmid B (2000a) Isolation and characterization of microsatellite loci in the bearded vulture (Gypaetus barbatus) and cross-amplification in three Old World vulture species. Molecular Ecology, 9, 2193–2195.

Gautschi B, Widmer A, Koella J (2000b) Isolation and characteriza- tion of microsatellite loci in the dice snake (Natrix tessellata). Molecular Ecology, 9, 2191–2193.

Hawley DM, Hanley D, Dhondt AA, Lovette IJ (2006) Molecular evidence for a founder effect in invasive house finch (Carpodacus mexicanus) populations experiencing an emergent disease epidemic. Molecular Ecology, 15, 263–275.

Hill GE (2002) A Red Bird in a Brown Bag: The Function and Evolution of Colorful Plumage in the House Finch. Oxford University Press, Oxford, UK.

Jamieson A, Taylor SCS (1997) Comparisons of three probability formulae for parentage exclusion. Animal Genetics, 28, 397–400.

Lindstedt E, Oh KP, Badyaev AV (2006) Ecological, social, and genetic contingency of extrapair behavior in a socially monog- mous bird. Journal of Avian Biology, 38, 214–238.

Oh KP, Badyaev AV (2006) Adaptive genetic complementarity in mate choice coexists with preference for elaborate sexual traits. Proceedings of the Royal Society B: Biological Sciences, 273, 1913–1919. Oh KP, Badyaev AV (2008) Evolution of adaptation and mate choice: parental relatedness affects expression of phenotypic vari- ance in a natural population. Evolutionary Biology, 35, 111–124.

Pritchard JK, Stephens M, Donnelly P (2000) Inference of population structure using multilocus genotype data. Genetics, 155, 945–959. Queller DC, Strassmann JE, Hughes CR (1993) Microsatellites and kinship. Trends in Ecology & Evolution, 8, 285–288.

Raymond M, Rousset F (1995) GenePop (version 1.2): population genetics software for exact tests and ecumenicism. Journal of Heredity, 86, 248–249.

Tenzer I, degli Ivanissevich S, Morgante M, Gessler C (1999) Identification of microsatellite markers and their application to population genetics of Venturia inaequalis. Phytopathology, 89, 748–753.

Veit RR, Lewis MA (1996) Dispersal, population growth, and the Allee effect: dynamics of the house finch invasion of eastern North America. American Naturalist, 148, 255–274.

Wang Z, Baker AJ, Hill GE, Edwards SV (2003) Reconciling actual and inferred population histories in the house finch (Carpodacus mexicanus) by AFLP analysis. Evolution, 57, 2852–2864.

Wootton JT (1987) Interspecific competition between introduced house finch populations and two associated passerine species. Oecologia, 71, 325–331.

doi:  10.1111/j.1755-0998.2009.02555.x

 $@$  2009 Blackwell Publishing Ltd

# Isolation and characterization of new microsatellite markers  for rose bitterlings, Rhodeus ocellatus

Y. SHIRAI,\* S. IKEDA† and S. TAJIMA‡ \*Kagawa Prefectural Research Institute for Environmental Sciences and Public Health, 5-3-105 Asahimachi, Takamatsu, 760-0065,  Japan, †Gene Research Center, Kagawa University, 2393, Ikenobe, Miki-cho, Kita-gun, Kagawa 761-0795, Japan, ‡Department of Life  Sciences, Kagawa University, 2393, Ikenobe, Miki-cho, Kita-gun, Kagawa 761-0795, Japan

## Abstract

The Japanese rose bitterling (Rhodeus ocellatus kurumeus because of hybridization and competition from an invasive alien subspecies (Rhodeus ocellatus ocellatus dinucleotide repeat specific polymerase chain reaction. The number of alleles per locus and R. o. kurumeus were lower than those in R. o. ocellatus. microsatellite markers were successfully cross-amplified in three Acheilognathinae species.

Keywords: Acheilognathinae, cross-species amplification, microsatellite markers, Rhodeus ocellatus, rose bitterling Received 24 April 2008; revision accepted 10 December 2008 Correspondence: Y. Shirai, Fax: +81-87-825-0408; E-mail: vg7552@pref.kagawa.lg.jp Two subspecies of the rose bitterling, Rhodeus ocellatus kurumeus (Teleostei:Cyprinidae) and R. o. ocellatus, inhabit Japan. Rhodeus o. kurumeus is endemic and facing extinction brought about by both hybridization and competition from an invasive alien subspecies, R. o. ocellatus. Rhodeus o. kurumeus is currently listed as a Critically Endangered species in the Red Data Book of the Japanese Ministry of the Environment. Clear and convenient identification of the two subspecies is needed to protect R. o. kurumeus from extinction. However, there have been no adequate molecular markers for R. ocellatus. We present the characterization of 11 new microsatellite markers for R. ocellatus and their applicability to three related species.

  

 



We amplified DNA fragments from the genomic DNA sample of R. o. kurumeus using dinucleotide repeat specific polymerase chain reaction (PCR) as described by Fisher et al. (1996). The PCR mixture consisted of  $20\,\mathrm{ng}$  genomic DNA, -anchored primer  $\scriptstyle({\mathrm{PCT}}_{6};$  $100\,\mathrm{nmol}\,5^{\prime}.$  KKVRVRVCTCTCTC TCTCT, or  $\mathrm{PCA}_{6};$  KKBRBRBCACACACACACA),  $800\,\upmu\mathbf{M}$  dNTPs,  $0.02\ \mathrm{U}/\upmu\mathrm{L}$  Taq DNA Polymerase (NEB), and  $1\times$  ThermoPol Buffer [NEB, 20 mm Tris-HCl,  $10\,\mathrm{{mm}\,(N H_{4})_{2}S O_{4}},$   Triton $.{\mathrm{X}}{\mathrm{-}}100,{\mathrm{pH}}\,8.8,$ ) in $10–\upmu\mathrm{L}$  $10\,\mathrm{{mm\,KCl}},2\,\mathrm{{mm\,MgSO}_{4}},0.1\%$  reaction volume. The temperature-cycle conditions were as follows: initial denaturation for 3 min at $94\,^{\circ}C;$  followed by 40 cycles of $25\,\mathrm{s}$  at $94\ {}^{\circ}\mathrm{C},60\,\mathrm{s}$  at $57\,^{\circ}C,$  and 45 s at $72\,^{\circ}C;$ ; and final extension for 7 min at  $72\,^{\circ}\mathrm{C}$ . The amplified DNA was cloned into pGEM-T easy (Promega) using TA cloning pro- tocol and the ligations were chemically transformed into DH5α (Toyobo). The insert fragments with enough length to design PCR primers were sequenced on a PRISM 3100 Automated Capillary DNA Sequencer (Applied Biosystems) using a DYEnamic ET Terminator Cycle Sequencing Kit (GE Healthcare Bio-Sciences). Eleven primer pairs were designed using Primer 3 software (Rozen & Skaletsky 2000) from the sequence data of 70 clones. 



To evaluate the applicability of the microsatellite markers, we used the genomic DNA samples prepared from 20 individuals each of both subspecies. Individuals of R. o. kurumeus and R. o. ocellatus were collected from two ponds in the same area (10 each) and one stream, respectively. The PCR mixture contained $20\,\mathrm{ng}$  of genomic DNA, $100\,\mathrm{nM}$  each of forward and reverse primer, $800\,\upmu\mathbf{M}$  dNTPs, $1.3\,\upmu\mathbf{M}$  FITC- dUTP (Roche Diagnostics),  $0.02\,\mathrm{U/mL}$  of Taq DNA Poly- merase (NEB), and  $1\times$ ThermoPol Buffer in  $10–\upmu\mathrm{L}$  reaction volume. The temperature-cycle conditions were as follows: initial denaturation for 3 min at $94\,^{\circ}C,$ ; followed by 30 cycles —, no amplification.



of 15 s at  $94\,^{\circ}C,$   $30\,\mathrm{s}$  at  $56\,^{\circ}{\cal C},$  and  $20\,\mathrm{s}$ at  $72\,^{\circ}{\cal C};$  and final extension for  $7\,\mathrm{min}$  at  $72\,^{\circ}\mathrm{C}$ . The amplified microsatellite markers were mixed with three volumes of formamide dye solution  $(95\%$  formamide,  $10\mathrm{\mm\EDTANa_{2}},\,10\mathrm{\mg/mL}$  bluedextran), heat denatured at  $94\,^{\circ}\mathrm{C}$  for  $3\,\mathrm{min}$  and run with MapMarker (BioVenture) on a  $5\%$  PAGE-PLUS (Amresco) — 8 M urea denaturing gel using a DSQ-1000 Slab Gel DNA Sequencer (Shimadzu Biotech). 

The characteristics of newly developed microsatellite markers are shown in Table 1. Estimations of the expected and observed heterozygosities and the assessment of Hardy– Weinberg equilibrium and linkage disequilibria were carried out using GenePop version 3.4 (Raymond & Rousset 1995). In R. o. kurumeus, five and six out of 11 microsatellite markers were monomorphic and polymorphic, respectively. Exact tests of deviation from Hardy–Weinberg equilibrium and estimates of linkage disequilibrium in the case of poly- morphic markers were not significant after sequential Bonferroni correction (Rice 1989). Conversely, in R. o. ocellatus, all the microsatellite markers were multi-allelic. The hetero- zygosities of all markers except RC364 were high, with numbers of alleles ranging from six to 12. Exact tests of deviation from Hardy–Weinberg equilibrium were significant in RC306, RC364 and RC688 after sequential Bonferroni correction. The deviation may result from the small sample size or the presence of null alleles. The contribution of the latter was analysed with Micro-Checker (van Oosterhout et al. 2004). The estimated frequencies of null alleles at RC306, RC364 and RC688 were 0.16, 0.38 and 0.14, respectively, indicating that the deviation is likely to be caused by null alleles. The linkage disequilibria in R. o. ocellatus were not significant in all locus pairs after sequential Bonferroni correction.



Twenty individuals of R. o. kurumeus were classified into 18 genotypes using a combination of six markers  $({\mathrm{RC}}236,$  292, 363, 364, 625 and 772). In addition, all the microsatellite markers except RC236 were cross-amplified successfully in three other Acheilognathinae species (Table 2). Therefore, these microsatellite markers are not only useful for research of the genetic structure of R. o. ocellatus and the conservation of R. o. kurumeus, but are also possibly applicable to other related endangered Acheilognathinae species such as Tanakia tanago.

## Acknowledgements

We would like to thank Y. Kanoh for the R. o. kurumeus sample and H. Kubota for the R. o. ocellatus, T. lanceolata, A. melanogaste and A. tabira samples.

## References

Fisher PJ, Gardner RC, Richardson TE (1996) Single locus micro- satellites isolated using $5^{\prime}$  anchored PCR. Nucleic Acids Research, 24, 4369–4371.

van Oosterhout C, Hutchinson WF, Wills DPM, Shipley P (2004) Micro-Checker: software for identifying and correcting geno- typing errors in microsatellite data. Molecular Ecology Notes, 4, 535–538.

Raymond M, Rousset F (1995) GenePop (version 1.2): a population genetics software for exact test and ecumenicism. Journal of Heredity, 86, 248–249.

Rice WR (1989) Analyzing tables of statistical tests. Evolution, 43, 223–225.

Rozen S, Skaletsky H (2000) Primer 3 on the WWW for general users and biologist programmers. In: Bioinformatics Methods and Protocols: Methods in Molecular Biology (eds Krawetz S, Misener S), pp. 365–386. Humana Press, Totowa, New Jersey.
# Cerebral venous thrombosis Associationsbetweendiseaseseverityandcardiacmarkers

Michelle C. Johansen, MD, Rebecca F. Gottesman, MD, PhD, and Victor C. Urrutia, MD Neurology: Clinical Practice Month 2019 vol. 00 no. 00 1-7 doi:10.1212/CPJ.0000000000000670



Correspondence Dr. Johansen mjohans3@jhmi.edu

## Abstract

Background Plasma cardiac troponin (cTn) elevation occurs in acute ischemic stroke and intracranial hemorrhage and can suggest a poor prognosis. Because acute cerebral venous thrombosis (CVT) might lead to venous stasis, which could result in cardiac stress, it is important to evaluate whether cTn elevation occurs in patients with CVT.

Methods Inpatients at Johns Hopkins Hospital from 2005 to 2015 meeting the following criteria were included: CVT (ICD-9 codes with ra- diologic conrmation) and available admission electrocardiogram (ECG) and cTn level. In regression models, presence of ECG abnormalities and cTn elevation  $(>0.06~\mathrm{ng/mL})$  were evaluated as dependent variables in separate models, with location and severity of CVT involvement as independent variables, adjusted for age, sex, and hypertension.

Results Of 81 patients with CVST, 53  $(66\%)$  met the inclusion criteria. Participants were, on average, aged 42 years, white  $(71\%).$ , and female  $(66\%)$ . The left transverse sinus was most commonly thrombosed  $(47\%).$ , with  $66\%$  having  ${>}2$  veins thrombosed. Twenty-two  $(41\%)$  had cTn elevation. Odds of cTn elevation increased per each additional vein thrombosed (adjusted OR 2.79,  $95\%$  CI [1.087.23]). Of those with deep venous involvement,  $37.5\%$  had cTn elevation compared with  $4.4\%$  without deep clots  $(p=0.02)$ . Venous infarction ( $\mathrm{{(n=15)}}$ ) was associated with a higher mean cTn (0.14 vs  $0.02\;\mathrm{ng/mL},p=0.009)$  and was predictive of a higher cTn in adjusted models  $\left(\beta=0.15,\,95\%\,\,\mathrm{CI}\,\,\big[0.06{-}0.25\big]\right)$ .

Conclusions In this single-center cohort study, markers of CVT severity were associated with increased odds of cTn elevation; further investigation is needed to elucidate causality and signiﬁcance.

Cerebral venous thrombosis (CVT) is distinct in the ﬁeld of cerebrovascular neurology, given its predilection for young adults with minimal comorbidity. The pathophysiology of brain injury from CVT is generally from (1) increased intracranial pressure (ICP) from impaired venous drainage resulting in neurologic dysfunction or (2) venous ischemia and/or hemor- rhage leading to focal brain parenchymal damage. Venous stasis can lead to elevated ICP, venous infarct, or hemorrhage, suggesting that the 2 mechanisms are likely synergistic, but the

There is increasing recognition and research regarding the importance of the heart-brain axis in stroke prognosis.

clinical decits resulting from either mechanism can lie along a spectrum ranging from mild disease to profound morbidity.

There is increasing recognition and research regarding the importance of the heart-brain axis in stroke prognosis. Patients with arterial ischemic stroke with elevated plasma cardiac troponin  $(\mathrm{cTn}).$ , compared with those without ele- vated  $\mathrm{cTn},$  have increased mortality, as do those with in- tracranial hemorrhage, independent of hemorrhage volume, sex, and age. In-hospital electrocardiogram (ECG) changes portend a poorer prognosis for both stroke types. Single measures of cTn are associated with incident venous thromboembolism, but whether such changes are seen in patients with CVT is unknown. As cTn is a marker of cardiac injury, it is plausible that turbulent and impaired venousow occurring in more severe CVT cases, such as those with higher degrees of clot burden, would have elevated cTn. The aim of this study is to describe possible cardiac changes, speciﬁcally ECG characteristics and cTn levels, seen in patients with CVT. We hypothesize that more severe fea- tures of CVT will be associated with abnormal ECG ﬁndings or cTn elevation.

## Methods

Study population Study inclusion criteria were admission to Johns Hopkins Hospital during a 10-year period (2005–2015) with a di- agnosis of CVT (ICD-9: 325, 437.6) conﬁrmed by in- patient cerebral imaging, with an available inpatient ECG and cTn drawn during the hospitalization. ECG is standard of care, but cTn was measured at the discretion of the clinical team. Cerebral imaging used to conrm diagnosis was either cerebral tomography angiography or MR angi- ography. The presence of venous infarction and venous hemorrhage was determined using radiology reports. Age was deﬁned in years at the time of admission. Race and ethnicity were self-reported at the time of admission. Hypertension was deﬁned as a medical history of hyper- tension. The study was single centered to enable consis- tent methodology.

ECG measures The following continuous ECG variables were collected: QT interval (ms), QRS duration (ms), ventricular rate (bpm), atrial rate (bpm), P axis (degrees), and R axis (degrees). The presence or absence of T-wave inversion, ST elevation, U waves,  $\mathrm{Q}$ waves, premature ventricular contractions, pre- mature atrial contractions, right bundle branch block, and/ or left bundle branch block was also determined. These variables were collected, by an investigator masked to CVT characteristics, from the clinical reports from during the inpatient hospitalization, with additional direct review and conﬁrmation of the ECG tracings by the study physi- cian (M.C.J.).



Cardiac troponin cTn was drawn during the inpatient stay, with the level measured by the Johns Hopkins clinical laboratory. Levels were recorded continuously and a positive cTn deﬁned as  $\geq0.06~\mathrm{ng/mL}$ .

CVT characteristics Imaging reports were reviewed masked to ECG data to determine the thrombus location and the individual clot count per patient. Speciﬁcally, the location of the venous clot was categorized as present or absent using the following designations: superior sagittal sinus, left transverse sinus (LTS), right transverse sinus, left sigmoid sinus, right sig- moid sinus, left internal jugular, right internal jugular, in- ferior sigmoid sinus, straight sinus (SS), cortical vein thrombosis (deﬁned as clot in the vein of Labbe or Trolard), deep venous thrombosis (dened as clot in the medullary and/or subependymal veins), and cavernous sinus throm- bosis. A large thrombus extending from one venous struc- ture to the adjacent venous structure was counted as present in both structures. Imaging reports were reviewed to de- termine the presence or absence of insular stroke, venous infarction, and venous hemorrhage, each. If the patient had more than one imaging procedure performed during in- patient admission, the scan resulting in diagnosis (usually the ﬁrst) was the one considered for analysis and denition of CVT characteristics.

Covariates Demographics and risk factor data were obtained from the medical record. Demographics considered were race and sex and age at diagnosis of CVT. Risk factor data collected were based on the literature and included a documented (elec- tronic medical record) medical history of type 2 diabetes, high cholesterol, coronary artery disease, atrialbrillation, previous history of stroke, previous history of intracerebral hemorrhage, sickle cell disease, and oral contraceptive use at the time of CVT event.

Statistical analysis Categorical variables are reported as percentages, whereas continuous variables are reported as mean values. Two- sample t tests with equal variance and  $\chi^{2}$ method were used for univariable analysis for continuous and categorical variables, respectively. Multivariable linear regression was used to estimate the associations between continuous ECG parameters (dependent variable) and markers of CVT severity (clot location, clot count, presence of thrombus in deep venous structures, insular stroke, venous hemorrhage, and venous infarct). Multivariable logistic regression was used to estimate the associations between dichotomous ECG parameters and markers of CVT se- verity. Similarly, regression models were used to estimate the associations between continuous cTn or dichotomous positive cTn (dependent variables) and markers of CVT severity. Regression models were then adjusted for age and sex (model 1) and additionally for hypertension (model 2). Given the small number of observations, only those demographics for which there was the highest sus- picion for confounding were included in the adjustment models.



Standard protocol approvals, registrations, and patient consents For this retrospective analysis, patient data were obtained from hospital records in compliance with the institutional ethical standards committee.

Data availability Anonymized data not published within this article will be made available by request from any qualiﬁed investigator.

## Results

Demographics Fifty-three patients met the inclusion criteria; they were predominantly white females who were middle aged at the time of CVT diagnosis (table 1). Patients had little cardiac comorbidity, with the most common risk factor being a his- tory of hypertension  $(26\%)$ . Seven patients  $(13\%)$  were using oral contraceptive pills at the time of diagnosis. The LTS was the most common location for thrombus  $(47\%).$ , with 10 patients having hemorrhage associated with the CVT and 8 having an insular stroke (table 1). ECG characteristics fell within the normal range with an average ventricular rate for patients of 78  $(\pm16.0)$  bpm at the time of ECG. Seven patients had evidence of T-wave inversion. Twenty-two patients had a positive troponin with the overall range of  (table 1). $0.06{-}0.93~\mathrm{ng/mL}$  ECG analysis There was no diﬀerence when comparing mean values or percentages of either the continuous or dichotomous ECG variables when considering these markers across patients with CVT with insular stroke, venous hemorrhage, or venous infarct. When considering the total clot count of patients with CVT, none of the ECG parameters were found to be signif- icantly associated in either univariable or adjusted models (age, sex, and hypertension) (tables 2 and 3).



cTn analysis Of the 53 participants, 22 met the criteria for cTn elevation  $(41\%)$ . When considering the continuous troponin level,

The pathophysiology of cerebral venous thrombosis is unique and distinct and recognized to be different from that of arterial thrombosis.

there was no diﬀerence in the mean troponin level among those with venous hemorrhage or insular stroke. Those with venous infarction  $\mathrm{(n}=15)$ ) had a higher mean cTn (0.14 vs 0.02,  $p\,=\,0.009)$  than did individuals without venous in- farction. Venous infarct was associated with a higher tro- ponin level among those of the same age,  $s\mathrm{ex},$  and hypertension history  $\left(\beta=0.15,\,95\%\,\mathrm{CI}\,\left[0.06{-}0.25\right]\right)$  (table 4). Of those with deep venous involvement (deep veins, SS;  $\mathrm{~n~}=\mathrm{~8~}$ ),  $37.5\%$  had cTn elevation compared with  $4.4\%$  without deep clots  $\left(p=0.02\right)$ . Patients with a higher number of thrombosed deep venous structures had a higher mean troponin in the ﬁnal adjusted model ( $\beta=0.10$  per additional thrombosed deep venous structure,  $95\%$  CI [0.01–0.19]); however, there was no association between the presence of thrombus (versus no thrombus) in any individual deep vein and a higher mean troponin. Those with an insular stroke had a higher troponin level compared with those without, but only after adjusting for age, sex, and hypertension his- tory  $\left(\beta=0.12,\,95\%\,\,\mathrm{CI}\,\,\left[0.00{-}0.24\right]\right)$ . Those with a venous infarct or venous hemorrhage had approximately 20 times the odds of a positive troponin compared with those who did not have these CVT characteristics, but this estimate, while signiﬁcant, is imprecise. Odds of cTn elevation in- creased per each additional vein thrombosed (OR 2.79,  $95\%$  CI [1.087.23]).

## Discussion

Among patients admitted to an academic center with a di- agnosis of CVT, characteristics of more severe CVT such as venous infarction, venous hemorrhage, involvement of the deep venous structures, or increasing clot count were asso- ciated with an elevation of cTn.

The pathophysiology of CVT is unique and distinct and rec- ognized to be diﬀerent from that of arterial thrombosis. Vaso- genic edema occurs earlier in venous strokes than in arterial strokes and is thought to be due to the early increase in venous pressure that occurs from venous occlusion. Continued increases in venous pressure lead to venous rupture, resulting in hemorrhage, or alternatively, progressive drops in cerebral perfusion pressure cross the ischemic threshold and result in strokes.

Troponin elevation has been repeatedly described in strokes of all etiologies, with a meta-analysis suggesting an  $18\%$ 

Right transverse sinus22 (42%) Left sigmoid sinus18 (34%) Right sigmoid sinus13 (25%) Right internal jugular10 (19%) Left internal jugular11 (21%) Inferior sigmoid sinus2 (4%) SS7 (13%) Cortical vein5 (9%) Deep venous3 (6%) Cavernous sinus2 (4%)

        

       

      

     

    

   

  

 



overall prevalence. Troponin release occurs in the setting of cardiac cell necrosis, and hence, it has a role in measuring myocardial ischemia. However, a study involving patients with subarachnoid hemorrhage and with a positive cTn and wall motion abnormalities on echocardiogram failed to demonstrate any abnormalities during coronary angio- gram, suggesting that myocardial ischemia alone is not suﬃcient to describe what leads to cTn elevation in these patients.

Adequate venous return to the heart is pivotal for sustained cardiac output and involves the careful balance of mean systemic pressure and right atrial pressure. Mean systemic pressure is aﬀected by blood volume and vascular tone, with the sympathetic nervous system playing an important role in regulation of vascular tone. Cerebral venous drainage is complex with the assumption that increases in right arterial pressure are transmitted through the jugular venous channel, but how this relates to systemic venous return is not well described. It is recognized that autor- egulatory mechanisms protect the brain from alterations in cerebral blood ﬂow, but how much this model applies in the event of a disrupted blood-brain barrier, such as what can occur in CVT, is unknown. Imaging studies suggest that patients with CVT have alterations of venous hemo- dynamics with increased mean blood ﬂow velocity and turbulence of venousow in attempting to recruit venous collaterals. It is possible that CVT through this altered turbulent venous return could result in either early myo- cyte necrosis or some degree of myocardial stretch resulting in a positive troponin.

Our associations with cTn were the strongest among those with deep venous clots. Cerebral deep venous thrombosis is strongly associated with dysfunction of the diencephalon, which can manifest clinically with symptoms of decreased consciousness or other brain stem signs. The presence of a clot in the deep venous system has been associated with an increased risk of death and dependency. It is possible that our reported association represents not only a ﬂow phe- nomenon but also a disruption to the diencephalon, which modulates the autonomic system and neurohormonal acti- vation and results in nonischemic myocyte necrosis. Auto- nomic changes secondary to perturbations of particular areas of the brain have been suggested as one potential mechanism for  $c\mathrm{Tn}$  elevation in ischemic stroke and hemorrhage. Al- though the number of participants with insular stroke was low, we notably did not ﬁnd an association between insular involvement and odds of elevated cTn or the mean level of patients cTn.



We have postulated some of the mechanisms by which pos- itive cTn may be seen in CVT, but we recognize that there are limitations to this study. The small size of the cohort may limit our ability to draw conclusions. This study being ob- servational was conducted at a single center to have consis- tent methodology in terms of the diagnostic evaluation of these patients. The data presented are useful to generate hypotheses that will inform future research. A larger study is warranted to draw conclusions. The ECG obtained on ad- mission and the ﬁrst image study after admission were used to deﬁne study parameters in patients with multiple tests over the duration of the admission. It may be that associations would change should later ECGs or imaging be used. The included patients were also a population in whom a troponin level was obtained at the discretion of the admission team, and this may introduce selection bias. As a result of the ret- rospective nature of this work, clinical neurologic deﬁcits as documented were not included, so this work is unable to speak to the relationship between clinical deﬁcits and cTn. Furthermore, because of the observational nature of this study, we cannot draw conclusions about causality, and it is likely that there are some residual confounders.



In conclusion, we have demonstrated that among patients with acute CVT with more severe characteristics, there is an in- creased odds of cTn and that this increases per additional ve- nous structure involved and with deep venous involvement. Althoughtheexactmechanismandclinicalramications ofsuch a ﬁnding remain to be elucidated, these associations may pos- sibly suggest alterations ofow dynamics that have a systemic eﬀect.

Author contributions M.C.Johansen:drafting/revisingthemanuscript,dataacquisition, studyconceptordesign,andanalysisorinterpretationofdata.R.F. Gottesman: drafting/revising the manuscript, study concept or design, and analysis or interpretation of data. V.C. Urrutia: drafting/revising the manuscript and study supervision.

Study funding M.C. Johansen: American Heart Association Mentored Clinical and Population Research Award #16MCPRP30350000, Na- tional Institutes of Health/ICTR (KL2); Rebecca F. Gottesman: National Institute on Aging (K24 AG052573).

Disclosure M.C. Johansen reports no disclosures. R.F. Gottesman serves as associate editor of Neurology. V.C. Urrutia-Member serves on an advisory board for Genentech, Inc and site PI for TIMELESS (Sponsor: Genentech, Inc). Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.

Publication history Received by Neurology: Clinical Practice December 10, 2018. Accepted in ﬁnal form March 6, 2019.

References 1.Saposnik G, Barinagarrementeria F, Brown RD Jr, et al. Diagnosis and management of cerebral venous thrombosis: a statement for healthcare professionals from the AmericanHeartAssociation/AmericanStrokeAssociation.Stroke 2011;42: 1158–1192.

2.Peddada  $\mathrm{K},$  Cruz-Flores S, Goldstein LB, et al. Ischemic stroke with troponin ele- vation: patient characteristics, resource utilization, and in-hospital outcomes. Cere- brovasc Dis 2016;42:213223.

3.Garrett MC, Komotar RJ, Starke RM, Doshi D, Otten ML, Connolly ES. Elevated troponin levels are predictive of mortality in surgical intracerebral hemorrhage patients. Neurocrit Care 2010;12:199203.

4.Manocha KK, Snipelisky D, Anavekar NS. Intracranial hemorrhage and deep T-wave inversions. JAMA Intern Med 2015;175:12231225.

5.Jiang B, Han X, Wang L, Dong  $\mathrm{Q.}$  Prognosis of early-stage continuous electrocar- diogram abnormalities on patients with acute ischemic stroke. J Stroke Cerebrovasc Dis 2015;24:17611767.

6.Folsom AR, Lutsey  $\mathrm{PL},$  Nambi V, et al. NT-proBNP, and venous thromboembolism: the longitudinal investigation of thromboembolism etiology (LITE). Vasc Med 2014; 19:33–41.

7.Gotoh M, Ohmoto T, Kuyama H. Experimental study of venous circulatory disturbance by dural sinus occlusion. Acta Neurochir (Wien) 1993;124: 120–126.

8.Kerr G, Ray G, Wu O, Stott DJ, Langhorne P. Elevated troponin after stroke: a sys- tematic review. Cerebrovasc Dis 2009;28:220–226.

9.Muthu V, Kozman  $\mathrm{H},$  Liu K, Smulyan H, Villarreal D. Cardiac troponins: bench to bedside interpretation in cardiac disease. Am J Med Sci 2014;347:331–337.

10.Banki N, Kopelnik A, Tung P, et al. Prospective analysis of prevalence, distribution, and rate of recovery of left ventricular systolic dysfunction in patients with sub- arachnoid hemorrhage. J Neurosurg 2006;105:15–20.

11.Banki NM, Kopelnik A, Dae MW, et al. Acute neurocardiogenic injury after sub- arachnoid hemorrhage. Circulation 2005;112:3314–3319.

12.Young DB. Control of cardiac output. San Rafael: Morgan & Claypool Life Sciences; 2010.

13.Schaller B. Physiology of cerebral venous bloodow: from experimental data in animals to normal function in humans. Brain Res Brain Res Rev 2004;46:243–260.

14.Schuchardt F, Hennemuth A, Schroeder  $\mathrm{L},$  et al. Acute cerebral venous thrombosis: three-dimensional visualization and quantiﬁcation of hemodynamic alterations using 4-dimensional ﬂow magnetic resonance imaging. Stroke 2017;48:671–677.

15.van den Bergh WM, van der Schaaf I, van Gijn J. The spectrum of presentations of venous infarction caused by deep cerebral vein thrombosis. Neurology 2005;65:192–196.

16.Girot M, Ferro JM, Canhao P, et al. Predictors of outcome in patients with cerebral venous thrombosis and intracerebral hemorrhage. Stroke 2007;38:337–342.

DOI 10.1212/CPJ.0000000000000670 Neurol Clin Pract  published online May 17, 2019 Michelle C. Johansen, Rebecca F. Gottesman and Victor C. Urrutia markers Cerebral venous thrombosis: Associations between disease severity and cardiac

### This information is current as of May 17, 2019

Neurology. All rights reserved. Print ISSN: 2163-0402. Online ISSN: 2163-0933.

since 2011, it is now a bimonthly with 6 issues per year. Copyright © 2019 American Academy of  Neurol Clin Pract is an official journal of the American Academy of Neurology. Published continuously
# Cerebral venous thrombosis Associationsbetweendiseaseseverityandcardiacmarkers

Michelle C. Johansen, MD, Rebecca F. Gottesman, MD, PhD, and Victor C. Urrutia, MD Neurology: Clinical Practice Month 2019 vol. 00 no. 00 1-7 doi:10.1212/CPJ.0000000000000670



Correspondence Dr. Johansen mjohans3@jhmi.edu

## Abstract

Background Plasma cardiac troponin (cTn) elevation occurs in acute ischemic stroke and intracranial hemorrhage and can suggest a poor prognosis. Because acute cerebral venous thrombosis (CVT) might lead to venous stasis, which could result in cardiac stress, it is important to evaluate whether cTn elevation occurs in patients with CVT.

Methods Inpatients at Johns Hopkins Hospital from 2005 to 2015 meeting the following criteria were included: CVT (ICD-9 codes with ra- diologic conrmation) and available admission electrocardiogram (ECG) and cTn level. In regression models, presence of ECG abnormalities and cTn elevation  $(>0.06~\mathrm{ng/mL})$  were evaluated as dependent variables in separate models, with location and severity of CVT involvement as independent variables, adjusted for age, sex, and hypertension.

Results Of 81 patients with CVST, 53  $(66\%)$  met the inclusion criteria. Participants were, on average, aged 42 years, white  $(71\%).$ , and female  $(66\%)$ . The left transverse sinus was most commonly thrombosed  $(47\%).$ , with  $66\%$  having  ${>}2$  veins thrombosed. Twenty-two  $(41\%)$  had cTn elevation. Odds of cTn elevation increased per each additional vein thrombosed (adjusted OR 2.79,  $95\%$  CI [1.087.23]). Of those with deep venous involvement,  $37.5\%$  had cTn elevation compared with  $4.4\%$  without deep clots  $(p=0.02)$ . Venous infarction ( $\mathrm{{(n=15)}}$ ) was associated with a higher mean cTn (0.14 vs  $0.02\;\mathrm{ng/mL},p=0.009)$  and was predictive of a higher cTn in adjusted models  $\left(\beta=0.15,\,95\%\,\,\mathrm{CI}\,\,\big[0.06{-}0.25\big]\right)$ .

Conclusions In this single-center cohort study, markers of CVT severity were associated with increased odds of cTn elevation; further investigation is needed to elucidate causality and signiﬁcance.

Cerebral venous thrombosis (CVT) is distinct in the ﬁeld of cerebrovascular neurology, given its predilection for young adults with minimal comorbidity. The pathophysiology of brain injury from CVT is generally from (1) increased intracranial pressure (ICP) from impaired venous drainage resulting in neurologic dysfunction or (2) venous ischemia and/or hemor- rhage leading to focal brain parenchymal damage. Venous stasis can lead to elevated ICP, venous infarct, or hemorrhage, suggesting that the 2 mechanisms are likely synergistic, but the

There is increasing recognition and research regarding the importance of the heart-brain axis in stroke prognosis.

clinical decits resulting from either mechanism can lie along a spectrum ranging from mild disease to profound morbidity.

There is increasing recognition and research regarding the importance of the heart-brain axis in stroke prognosis. Patients with arterial ischemic stroke with elevated plasma cardiac troponin  $(\mathrm{cTn}).$ , compared with those without ele- vated  $\mathrm{cTn},$  have increased mortality, as do those with in- tracranial hemorrhage, independent of hemorrhage volume, sex, and age. In-hospital electrocardiogram (ECG) changes portend a poorer prognosis for both stroke types. Single measures of cTn are associated with incident venous thromboembolism, but whether such changes are seen in patients with CVT is unknown. As cTn is a marker of cardiac injury, it is plausible that turbulent and impaired venousow occurring in more severe CVT cases, such as those with higher degrees of clot burden, would have elevated cTn. The aim of this study is to describe possible cardiac changes, speciﬁcally ECG characteristics and cTn levels, seen in patients with CVT. We hypothesize that more severe fea- tures of CVT will be associated with abnormal ECG ﬁndings or cTn elevation.

## Methods

Study population Study inclusion criteria were admission to Johns Hopkins Hospital during a 10-year period (2005–2015) with a di- agnosis of CVT (ICD-9: 325, 437.6) conﬁrmed by in- patient cerebral imaging, with an available inpatient ECG and cTn drawn during the hospitalization. ECG is standard of care, but cTn was measured at the discretion of the clinical team. Cerebral imaging used to conrm diagnosis was either cerebral tomography angiography or MR angi- ography. The presence of venous infarction and venous hemorrhage was determined using radiology reports. Age was deﬁned in years at the time of admission. Race and ethnicity were self-reported at the time of admission. Hypertension was deﬁned as a medical history of hyper- tension. The study was single centered to enable consis- tent methodology.

ECG measures The following continuous ECG variables were collected: QT interval (ms), QRS duration (ms), ventricular rate (bpm), atrial rate (bpm), P axis (degrees), and R axis (degrees). The presence or absence of T-wave inversion, ST elevation, U waves,  $\mathrm{Q}$ waves, premature ventricular contractions, pre- mature atrial contractions, right bundle branch block, and/ or left bundle branch block was also determined. These variables were collected, by an investigator masked to CVT characteristics, from the clinical reports from during the inpatient hospitalization, with additional direct review and conﬁrmation of the ECG tracings by the study physi- cian (M.C.J.).



Cardiac troponin cTn was drawn during the inpatient stay, with the level measured by the Johns Hopkins clinical laboratory. Levels were recorded continuously and a positive cTn deﬁned as  $\geq0.06~\mathrm{ng/mL}$ .

CVT characteristics Imaging reports were reviewed masked to ECG data to determine the thrombus location and the individual clot count per patient. Speciﬁcally, the location of the venous clot was categorized as present or absent using the following designations: superior sagittal sinus, left transverse sinus (LTS), right transverse sinus, left sigmoid sinus, right sig- moid sinus, left internal jugular, right internal jugular, in- ferior sigmoid sinus, straight sinus (SS), cortical vein thrombosis (deﬁned as clot in the vein of Labbe or Trolard), deep venous thrombosis (dened as clot in the medullary and/or subependymal veins), and cavernous sinus throm- bosis. A large thrombus extending from one venous struc- ture to the adjacent venous structure was counted as present in both structures. Imaging reports were reviewed to de- termine the presence or absence of insular stroke, venous infarction, and venous hemorrhage, each. If the patient had more than one imaging procedure performed during in- patient admission, the scan resulting in diagnosis (usually the ﬁrst) was the one considered for analysis and denition of CVT characteristics.

Covariates Demographics and risk factor data were obtained from the medical record. Demographics considered were race and sex and age at diagnosis of CVT. Risk factor data collected were based on the literature and included a documented (elec- tronic medical record) medical history of type 2 diabetes, high cholesterol, coronary artery disease, atrialbrillation, previous history of stroke, previous history of intracerebral hemorrhage, sickle cell disease, and oral contraceptive use at the time of CVT event.

Statistical analysis Categorical variables are reported as percentages, whereas continuous variables are reported as mean values. Two- sample t tests with equal variance and  $\chi^{2}$ method were used for univariable analysis for continuous and categorical variables, respectively. Multivariable linear regression was used to estimate the associations between continuous ECG parameters (dependent variable) and markers of CVT severity (clot location, clot count, presence of thrombus in deep venous structures, insular stroke, venous hemorrhage, and venous infarct). Multivariable logistic regression was used to estimate the associations between dichotomous ECG parameters and markers of CVT se- verity. Similarly, regression models were used to estimate the associations between continuous cTn or dichotomous positive cTn (dependent variables) and markers of CVT severity. Regression models were then adjusted for age and sex (model 1) and additionally for hypertension (model 2). Given the small number of observations, only those demographics for which there was the highest sus- picion for confounding were included in the adjustment models.



Standard protocol approvals, registrations, and patient consents For this retrospective analysis, patient data were obtained from hospital records in compliance with the institutional ethical standards committee.

Data availability Anonymized data not published within this article will be made available by request from any qualiﬁed investigator.

## Results

Demographics Fifty-three patients met the inclusion criteria; they were predominantly white females who were middle aged at the time of CVT diagnosis (table 1). Patients had little cardiac comorbidity, with the most common risk factor being a his- tory of hypertension  $(26\%)$ . Seven patients  $(13\%)$  were using oral contraceptive pills at the time of diagnosis. The LTS was the most common location for thrombus  $(47\%).$ , with 10 patients having hemorrhage associated with the CVT and 8 having an insular stroke (table 1). ECG characteristics fell within the normal range with an average ventricular rate for patients of 78  $(\pm16.0)$  bpm at the time of ECG. Seven patients had evidence of T-wave inversion. Twenty-two patients had a positive troponin with the overall range of  (table 1). $0.06{-}0.93~\mathrm{ng/mL}$  ECG analysis There was no diﬀerence when comparing mean values or percentages of either the continuous or dichotomous ECG variables when considering these markers across patients with CVT with insular stroke, venous hemorrhage, or venous infarct. When considering the total clot count of patients with CVT, none of the ECG parameters were found to be signif- icantly associated in either univariable or adjusted models (age, sex, and hypertension) (tables 2 and 3).



cTn analysis Of the 53 participants, 22 met the criteria for cTn elevation  $(41\%)$ . When considering the continuous troponin level,

The pathophysiology of cerebral venous thrombosis is unique and distinct and recognized to be different from that of arterial thrombosis.

there was no diﬀerence in the mean troponin level among those with venous hemorrhage or insular stroke. Those with venous infarction  $\mathrm{(n}=15)$ ) had a higher mean cTn (0.14 vs 0.02,  $p\,=\,0.009)$  than did individuals without venous in- farction. Venous infarct was associated with a higher tro- ponin level among those of the same age,  $s\mathrm{ex},$  and hypertension history  $\left(\beta=0.15,\,95\%\,\mathrm{CI}\,\left[0.06{-}0.25\right]\right)$  (table 4). Of those with deep venous involvement (deep veins, SS;  $\mathrm{~n~}=\mathrm{~8~}$ ),  $37.5\%$  had cTn elevation compared with  $4.4\%$  without deep clots  $\left(p=0.02\right)$ . Patients with a higher number of thrombosed deep venous structures had a higher mean troponin in the ﬁnal adjusted model ( $\beta=0.10$  per additional thrombosed deep venous structure,  $95\%$  CI [0.01–0.19]); however, there was no association between the presence of thrombus (versus no thrombus) in any individual deep vein and a higher mean troponin. Those with an insular stroke had a higher troponin level compared with those without, but only after adjusting for age, sex, and hypertension his- tory  $\left(\beta=0.12,\,95\%\,\,\mathrm{CI}\,\,\left[0.00{-}0.24\right]\right)$ . Those with a venous infarct or venous hemorrhage had approximately 20 times the odds of a positive troponin compared with those who did not have these CVT characteristics, but this estimate, while signiﬁcant, is imprecise. Odds of cTn elevation in- creased per each additional vein thrombosed (OR 2.79,  $95\%$  CI [1.087.23]).

## Discussion

Among patients admitted to an academic center with a di- agnosis of CVT, characteristics of more severe CVT such as venous infarction, venous hemorrhage, involvement of the deep venous structures, or increasing clot count were asso- ciated with an elevation of cTn.

The pathophysiology of CVT is unique and distinct and rec- ognized to be diﬀerent from that of arterial thrombosis. Vaso- genic edema occurs earlier in venous strokes than in arterial strokes and is thought to be due to the early increase in venous pressure that occurs from venous occlusion. Continued increases in venous pressure lead to venous rupture, resulting in hemorrhage, or alternatively, progressive drops in cerebral perfusion pressure cross the ischemic threshold and result in strokes.

Troponin elevation has been repeatedly described in strokes of all etiologies, with a meta-analysis suggesting an  $18\%$ 

Right transverse sinus22 (42%) Left sigmoid sinus18 (34%) Right sigmoid sinus13 (25%) Right internal jugular10 (19%) Left internal jugular11 (21%) Inferior sigmoid sinus2 (4%) SS7 (13%) Cortical vein5 (9%) Deep venous3 (6%) Cavernous sinus2 (4%)

        

       

      

     

    

   

  

 



overall prevalence. Troponin release occurs in the setting of cardiac cell necrosis, and hence, it has a role in measuring myocardial ischemia. However, a study involving patients with subarachnoid hemorrhage and with a positive cTn and wall motion abnormalities on echocardiogram failed to demonstrate any abnormalities during coronary angio- gram, suggesting that myocardial ischemia alone is not suﬃcient to describe what leads to cTn elevation in these patients.

Adequate venous return to the heart is pivotal for sustained cardiac output and involves the careful balance of mean systemic pressure and right atrial pressure. Mean systemic pressure is aﬀected by blood volume and vascular tone, with the sympathetic nervous system playing an important role in regulation of vascular tone. Cerebral venous drainage is complex with the assumption that increases in right arterial pressure are transmitted through the jugular venous channel, but how this relates to systemic venous return is not well described. It is recognized that autor- egulatory mechanisms protect the brain from alterations in cerebral blood ﬂow, but how much this model applies in the event of a disrupted blood-brain barrier, such as what can occur in CVT, is unknown. Imaging studies suggest that patients with CVT have alterations of venous hemo- dynamics with increased mean blood ﬂow velocity and turbulence of venousow in attempting to recruit venous collaterals. It is possible that CVT through this altered turbulent venous return could result in either early myo- cyte necrosis or some degree of myocardial stretch resulting in a positive troponin.

Our associations with cTn were the strongest among those with deep venous clots. Cerebral deep venous thrombosis is strongly associated with dysfunction of the diencephalon, which can manifest clinically with symptoms of decreased consciousness or other brain stem signs. The presence of a clot in the deep venous system has been associated with an increased risk of death and dependency. It is possible that our reported association represents not only a ﬂow phe- nomenon but also a disruption to the diencephalon, which modulates the autonomic system and neurohormonal acti- vation and results in nonischemic myocyte necrosis. Auto- nomic changes secondary to perturbations of particular areas of the brain have been suggested as one potential mechanism for  $c\mathrm{Tn}$  elevation in ischemic stroke and hemorrhage. Al- though the number of participants with insular stroke was low, we notably did not ﬁnd an association between insular involvement and odds of elevated cTn or the mean level of patients cTn.



We have postulated some of the mechanisms by which pos- itive cTn may be seen in CVT, but we recognize that there are limitations to this study. The small size of the cohort may limit our ability to draw conclusions. This study being ob- servational was conducted at a single center to have consis- tent methodology in terms of the diagnostic evaluation of these patients. The data presented are useful to generate hypotheses that will inform future research. A larger study is warranted to draw conclusions. The ECG obtained on ad- mission and the ﬁrst image study after admission were used to deﬁne study parameters in patients with multiple tests over the duration of the admission. It may be that associations would change should later ECGs or imaging be used. The included patients were also a population in whom a troponin level was obtained at the discretion of the admission team, and this may introduce selection bias. As a result of the ret- rospective nature of this work, clinical neurologic deﬁcits as documented were not included, so this work is unable to speak to the relationship between clinical deﬁcits and cTn. Furthermore, because of the observational nature of this study, we cannot draw conclusions about causality, and it is likely that there are some residual confounders.



In conclusion, we have demonstrated that among patients with acute CVT with more severe characteristics, there is an in- creased odds of cTn and that this increases per additional ve- nous structure involved and with deep venous involvement. Althoughtheexactmechanismandclinicalramications ofsuch a ﬁnding remain to be elucidated, these associations may pos- sibly suggest alterations ofow dynamics that have a systemic eﬀect.

Author contributions M.C.Johansen:drafting/revisingthemanuscript,dataacquisition, studyconceptordesign,andanalysisorinterpretationofdata.R.F. Gottesman: drafting/revising the manuscript, study concept or design, and analysis or interpretation of data. V.C. Urrutia: drafting/revising the manuscript and study supervision.

Study funding M.C. Johansen: American Heart Association Mentored Clinical and Population Research Award #16MCPRP30350000, Na- tional Institutes of Health/ICTR (KL2); Rebecca F. Gottesman: National Institute on Aging (K24 AG052573).

Disclosure M.C. Johansen reports no disclosures. R.F. Gottesman serves as associate editor of Neurology. V.C. Urrutia-Member serves on an advisory board for Genentech, Inc and site PI for TIMELESS (Sponsor: Genentech, Inc). Full disclosure form information provided by the authors is available with the full text of this article at Neurology.org/cp.

Publication history Received by Neurology: Clinical Practice December 10, 2018. Accepted in ﬁnal form March 6, 2019.

References 1.Saposnik G, Barinagarrementeria F, Brown RD Jr, et al. Diagnosis and management of cerebral venous thrombosis: a statement for healthcare professionals from the AmericanHeartAssociation/AmericanStrokeAssociation.Stroke 2011;42: 1158–1192.

2.Peddada  $\mathrm{K},$  Cruz-Flores S, Goldstein LB, et al. Ischemic stroke with troponin ele- vation: patient characteristics, resource utilization, and in-hospital outcomes. Cere- brovasc Dis 2016;42:213223.

3.Garrett MC, Komotar RJ, Starke RM, Doshi D, Otten ML, Connolly ES. Elevated troponin levels are predictive of mortality in surgical intracerebral hemorrhage patients. Neurocrit Care 2010;12:199203.

4.Manocha KK, Snipelisky D, Anavekar NS. Intracranial hemorrhage and deep T-wave inversions. JAMA Intern Med 2015;175:12231225.

5.Jiang B, Han X, Wang L, Dong  $\mathrm{Q.}$  Prognosis of early-stage continuous electrocar- diogram abnormalities on patients with acute ischemic stroke. J Stroke Cerebrovasc Dis 2015;24:17611767.

6.Folsom AR, Lutsey  $\mathrm{PL},$  Nambi V, et al. NT-proBNP, and venous thromboembolism: the longitudinal investigation of thromboembolism etiology (LITE). Vasc Med 2014; 19:33–41.

7.Gotoh M, Ohmoto T, Kuyama H. Experimental study of venous circulatory disturbance by dural sinus occlusion. Acta Neurochir (Wien) 1993;124: 120–126.

8.Kerr G, Ray G, Wu O, Stott DJ, Langhorne P. Elevated troponin after stroke: a sys- tematic review. Cerebrovasc Dis 2009;28:220–226.

9.Muthu V, Kozman  $\mathrm{H},$  Liu K, Smulyan H, Villarreal D. Cardiac troponins: bench to bedside interpretation in cardiac disease. Am J Med Sci 2014;347:331–337.

10.Banki N, Kopelnik A, Tung P, et al. Prospective analysis of prevalence, distribution, and rate of recovery of left ventricular systolic dysfunction in patients with sub- arachnoid hemorrhage. J Neurosurg 2006;105:15–20.

11.Banki NM, Kopelnik A, Dae MW, et al. Acute neurocardiogenic injury after sub- arachnoid hemorrhage. Circulation 2005;112:3314–3319.

12.Young DB. Control of cardiac output. San Rafael: Morgan & Claypool Life Sciences; 2010.

13.Schaller B. Physiology of cerebral venous bloodow: from experimental data in animals to normal function in humans. Brain Res Brain Res Rev 2004;46:243–260.

14.Schuchardt F, Hennemuth A, Schroeder  $\mathrm{L},$  et al. Acute cerebral venous thrombosis: three-dimensional visualization and quantiﬁcation of hemodynamic alterations using 4-dimensional ﬂow magnetic resonance imaging. Stroke 2017;48:671–677.

15.van den Bergh WM, van der Schaaf I, van Gijn J. The spectrum of presentations of venous infarction caused by deep cerebral vein thrombosis. Neurology 2005;65:192–196.

16.Girot M, Ferro JM, Canhao P, et al. Predictors of outcome in patients with cerebral venous thrombosis and intracerebral hemorrhage. Stroke 2007;38:337–342.

DOI 10.1212/CPJ.0000000000000670 Neurol Clin Pract  published online May 17, 2019 Michelle C. Johansen, Rebecca F. Gottesman and Victor C. Urrutia markers Cerebral venous thrombosis: Associations between disease severity and cardiac

### This information is current as of May 17, 2019

Neurology. All rights reserved. Print ISSN: 2163-0402. Online ISSN: 2163-0933.

since 2011, it is now a bimonthly with 6 issues per year. Copyright © 2019 American Academy of  Neurol Clin Pract is an official journal of the American Academy of Neurology. Published continuously
A 5.2 mW IEEE 802.15.6 HBC Standard Compatible Transceiver With Power Efﬁcient Delay-Locked-Loop Based BPSK Demodulator

Hyunwoo Cho, Student Member, IEEE, Hyungwoo Lee, Student Member, IEEE, Joonsung Bae, Member, IEEE, and Hoi-Jun Yoo, Fellow, IEEE

Abstract—A Low-power IEEE 802.15.6 human body communi- cation (HBC) fully compatible transceiver is implemented in 130 nm CMOS process. In this work, the proposed HBC transceiver satisfying all the standard requirements has four key features for low power consumption which includes: 1) low-power analog ac- tive ﬁlters for TX spectral mask:  $\mathbf{30\%}$  power reduction; 2) de- layed locked loop (DLL) based BPSK receiver with the S/H oper- ation for turning off unnecessary blocks:  $40\%$  power reduction; 3) low-power mode (LP-mode) receiver with the received signal strength indicator (RSSI) output data:  $\mathbf{50\%}$  power reduction; and 4) reconﬁgurable LNA with RSSI output data:  $\mathbf{60\%}$  power reduc- tion. As a result, the proposed transceiver can fully satisfy the HBC standard requirements while consuming  $\pmb{5.2}\ \mathbf{mW}$  from the  $\mathbf{1.2\V}$  supply.

Index Terms—Costas loop, delayed-locked loop (DLL), human body communication (HBC), IEEE 802.15.6, received signal strength indicator (RSSI), spectral mask, standard, wireless body area network (WBAN).

I. INTRODUCTION R ECENTLY, wireless body area network (WBAN) is getting more and more attention in the emerging appli- cations which combines healthcare and consumer electronics around the human body. The major design challenge associated with the WBAN is to extend the lifetime of the WBAN devices under limited energy source. Human body communication (HBC) which uses the human body as a communication channel is considered as a power-efﬁcient wireless communication solution of the WBAN because high conductivity of the human body in a low frequency band enables low power communica- tion. Since the HBC was ﬁrstly presented in [1], the physical communication principle and a variety of power efﬁcient HBC transceivers have been presented [2]–[14], and eventually in 2012 the WBAN is standardized as IEEE 802.15.6 [15]. However, fully satisfying all requirements of HBC standard (Fig. 1) while consuming low power for long battery life is very difﬁcult because of 1) transmitter (TX) mask with low spectrum ﬂoor, sharp slope and wide frequency bandwidth, 2) low sensitivity, and 3) received energy detection capability for PHY/MAC protocol.



The previous HBC transceiver was proposed in 2013 [9], but it was not optimized for low power consumption. In the TX, the power-consuming high-order digital ﬁlter which consists of high speed DAC with high speed clock generation  $(>\!840\,\mathrm{MHz})$  to avoid aliasing effect was adopted to satisfy the mask require- ments. In the RX, the power-hungry VCO-based BPSK demod- ulator with I (in phase) and Q (quadrature phase) dual data paths for coherent receiving was employed in RX, which requires sig- niﬁcant power consumption. In addition, the energy detector was not implemented which is an essential block for PHY/MAC protocol.

In this paper, we propose a low power and fully IEEE 802.  $15.6\,\mathrm{HBC}$  compatible transceiver satisfying all the tight speciﬁ- cations by adopting 4 key techniques. First, in the TX, an analog active ﬁlter rather than the high speed DAC reduces the power consumption by  $30\%$  because bandwidth requirement of analog active ﬁlter is much lower than the DAC in [9]. Second, the delay-locked-loop (DLL) based BPSK demodulator mitigates  $40\%$  of power consumption by turning off synchronization cir- cuits in Q-path data path with the help of sample and hold (S/H) circuit which retains the control voltage of the DLL. Third, the received signal strength indicator (RSSI), which measures the received signal power to obtain the energy detection ca- pability for the energy-efﬁcient PHY/MAC operation, enables the operation of the low-power mode (LP-mode) receiver com- posed of ampliﬁers used in RSSI and digital demodulator while turning off all blocks of the DLL-based BPSK receiver with  $50\%$  power reduction. At last, by utilizing the information of the input signal power, the noise and linearity performance of the LNA can be adaptively controlled with the  $60\%$  reduced bias current consumption.

The rest of this paper is organized as follows. In the Section II, the overall architecture of the proposed HBC transceiver will be introduced. In the Section III and IV, the detail design of the transmitter with the analog active ﬁlter and DLL-based BPSK receiver will be explained, respectively. In Section V, the low power schemes including LP-mode receiver and reconﬁgurable LNA will be presented with the RSSI design. After illustrating the measurement results in Section VI, the conclusion will be summarized in Section VII.

Fig. 2 shows the overall transceiver architecture which is composed of a transmitter with a spectral mask ﬁlter, a direct conversion receiver with a BPSK synchronizer, a RSSI and a digital baseband (PHY and MAC). In the transmitter, the base- band data from the PHY/MAC is modulated with 21 MHz car- rier frequency. The modulation is the frequency selective dig- ital transmission (FSDT) which is the same with the rectangular wave modulated BPSK. The spectral mask ﬁlter suppresses the low frequency band and high frequency band of the modulated data, and the driver transmits the data through the human body. In the receiver, the direct conversion receiver with the BPSK synchronizer recovers the received data from the LNA. The dif- ferential input LNA is adopted to generate differential output without noise ﬁgure growth which is essential for the direct conversion receiver conﬁguration. Since the TX is disconnected when the receiver is operated, it does not generate LNA input mismatch. I and Q data paths of the direct conversion receiver consists of active mixers, baseband ampliﬁers, baseband low- pass ﬁlters, and limiting ampliﬁers. The DLL-based BPSK syn- chronizer adjusts its local RX clock signal to the rising and falling edge of the input data. Especially compared with the VCO-based architecture [15], [16], the DLL-based architecture can promptly synchronize its clock with the input data without any stability problem. Moreover, the proposed synchronizer in Q data path including mixer, baseband ampliﬁer, low pass ﬁlter and limiting ampliﬁer is disabled to signiﬁcantly decrease its power consumption when the phase synchronization loop is sta- bilized. The RSSI is composed of a seven stage limiting ampli- ﬁer with seven rectiﬁers and 8-bit single-slope ADC. The RSSI measures the received input power with a range of $-100$  dBm to  $-10\:\mathrm{dBm}$ . Based on the output of the RSSI, the operation mode of the LNA is controlled to optimize its consuming power while satisfying the sensitivity requirements, resulting in  $60\%$  power

 



CHO et al.

reduction. In addition, to further reduce the power consumption when the input power is larger than $-35~\mathrm{dBm}$ , the direct con- version receiver circuits are turned off, and only the LP-mode receiver with the limiting ampliﬁer and digital demodulator for BPSK is turned on to recover the data. In the LP-mode receiver, over  $50\%$  of power consumption can be reduced.

quired to remove the harmonic frequency and alias terms, which leads to the signiﬁcant power consumption.

Fig. 3(b) shows the proposed transmitter architecture [13] with the low-power consuming analog active ﬁlters. Compared with the high speed DAC and digital ﬁlter, the analog active ﬁlter consumes much lower power because the OTA bandwidth requirement in the analog ﬁlter is much lower than the DAC. Since the spectral mask is falling at the  $23.625\,\mathrm{MHz}$  and the end frequency is  $400\,\mathrm{MHz},$ , the 3 dB frequency and gain bandwidth should be large than  $23.625~\mathrm{MHz}$  and  $400~\mathrm{MHz},$ , respectively. In this work, we designed the OTA 3 dB bandwidth as  $100\,\mathrm{MHz}$  which is larger than the  $23.625\;\mathrm{MHz}$ . Compared with the DAC which requires at least  $800\,\mathrm{MHz}$  sampling frequency, the analog ﬁlter requires only  $100\,\mathrm{MHz}$  bandwidth, and this bandwidth re- duction leads to  $30\%$  power reduction in the transmitter design. Although the proposed analog active ﬁlters require off-chip dig- itally controlled tuning to compensate the frequency character- istics variation caused by the process variation, it takes small cost because the tuning is required only at the ﬁrst time and the process variation can be avoided by using post layout sim- ulation and common centroid layout technique. The proposed transmitter is composed of the level shifter, analog active ﬁl- ters, driver and off-chip analog ﬁlters. The signal ﬂow in each stage in the transmitter will be explained in Fig. 4. Since the data is modulated with the rectangular wave swinging from  $0\mathrm{~V~}$  to  $1.2\;\mathrm{V},$  there are excessive DC power and harmonics in high fre- quency. Second, the data after the level shifter which reduces the swing range in order to avoid the non-linear effect in the following analog circuits is attenuated by 6th-order low-pass ﬁlter and 9th-order high-pass ﬁlter (Fig. 4(b)). The excessive spectrum power in DC and high frequency band is suppressed under the mask, but the non-linear characteristics of the analog active ﬁlter and driver increases the sideband spectrum, which is called spectrum regrowth effect (Fig. 4(c)). The amount of

 



CHO et al.

Fig. 9. Active mixer and its replica bias circuit.

Fig. 10. Delay circuit with the linear bias control circuit.

this increased low frequency spectrum power cannot be exactly predicted due to the PVT variation, so we adopt the simple R-C off-chip 2nd-order high-pass ﬁlter to additionally attenuate the low frequency power. After the off-chip 2nd-order high-pass ﬁlter, all the excessive spectrum power is removed, and ﬁnally, the output spectrum can satisfy the spectral mask requirement (Fig. 4(d)).

Fig. 5 shows the circuit of the 3rd-order active low-pass ﬁlter and high pass ﬁlter. The active ﬁlter adopts the multiple feed- back architecture which can obtain the $-60$  dB/decade slope with only one OTA. The OTA has folded cascade conﬁguration with the frequency compensation technique [14].

IV. RECEIVER DESIGN

# A. DLL-Based BPSK Receiver

Fig. 6 shows the analysis of the DLL-based BPSK receiver. In the BPSK receiver, the local clock synchronization path, which is called Costas loop, is essential. The receiver should have I path and Q path to calculate the phase difference between the input data and local clock by multiplying output of both paths. Since the multiplied output of I and Q path is proportional to , the phase difference can be zero by composing the syn- chronization loop to make thezero. In the conventional works [16], [17], the Costas loop uses the VCO to calibrate the input signal and local clock, which is quite similar with the PLL. Such implementations, however, may have stability concerns. In this work, we proposed a DLL-based Costas loop, which avoid the stability issues associated with 2nd (or higher order) phase-locked loops. Since the frequency of local clock is not changed with respect to the control voltage, the delay cell control voltage can be held with the help of sample and hold buffer while turning off the  $\mathsf Q$  path blocks, reducing  $40\%$  of power consumption. The phase error due to the  $\mathrm{S/H}$  buffer offset ( $^{<2}$  degrees) is negligible. In addition, the clock difference be- tween the TX and RX can be calibrated in the packet domain. In the HBC standard packet, the pilot which is composed of

 



Fig. 16. Reconﬁgurable LNA operation.

simple 0,1,0,1 for clock calibration must be inserted if the pay- load length is longer than 64 bytes. Since the clock drift of the crystal oscillator is usually under 10 ppm and it must be smaller than  $100\;\mathrm{ppm}$  in HBC standard, the clock offset can be period- ically calibrated at the pilot time without BER degradation.

Fig. 7 shows the detailed block diagram of the proposed DLL-based BPSK receiver. I path and Q path are composed of mixer buffer, mixer, baseband ﬁlter, and limiting ampliﬁer. The voltage to current converter and a capacitor make the 1st-order loop ﬁlter, and S/H buffer with a switch is employed for holding the control voltage while turning off the Q path blocks. The delayed 21 MHz reference clock signal through the delay cell in Costas loop works as a local clock of the RX.

Fig. 19. TX output power measurement results.

B. Circuit Design Fig. 8 shows the schematic of the reconﬁgurable LNA [8]. The common gate LNA with the-boosting cross coupled NMOS is adopted for low noise characteristics. The size of the input NMOSandare controlled by 6 bit control code to adaptively change the LNA performance for low-power con- sumption which will be explained in Section V, and the noise ﬁgure and the 1 dB compression point value in accordance with the control code is shown in Fig. 8.



Fig. 9 shows the schematic of the active mixer and replica bias circuit. The active mixer has a simple Gilbert mixer conﬁgura- tion, and resistoris added to reducing the switching noise and increasing the linearity of mixer. The resistorand are composing the replica bias circuit. It has a same conﬁg- uration of the half circuit of the Gilbert mixer, but resistor value is four times larger and size of transistor is four times smaller than the original ones, reducing the current consumption of the replica bias circuit by  $25\%$ .



Fig. 10 presents the delay cell and delay control bias cir- cuit. The delay cell is composed of six stages inverter based unit delay cells whose bias current is supplied by the delay con- trol bias circuit. In the bias circuit, the output current is linearly changed by the input control voltage. The two differen- tial MOS pairsremove the 2nd-order harmonics to increase the linearity while consuming more power consump- tion. Fig. 11 shows the measurement results of the delay time of the delay cell in accordance with the control voltageand bias current.

V. RSSI AND LOW-POWER RECEIVER

# A. RSSI Design

In the PHY/MAC protocol for multi-nodes operation, the en- ergy detection block is an essential. Fig. 12 shows the principle and architecture of the RSSI for energy detection [18]–[20]. The RSSI is composed of seven ampliﬁers, seven rectiﬁers, current adder with low-pass ﬁlter and ADC. Since the number of the ampliﬁers decides the dynamic range of the RSSI and we need  $-10$  dBm to $-100\:\mathrm{dBm}$  input dynamic range to sense the HBC input signal power, seven ampliﬁers with the voltage gain of 15 dB and seven rectiﬁers are adopted. Each ampliﬁer and rectiﬁer covers the different input power range, and we can achieve wide dynamic range by adding all rectiﬁers' results.

 



Fig. 13 shows the schematic of the rectiﬁer and current adder. The output current of each of the rectiﬁer with the size ratio of k can be expressed as (1) Since the output current is proportional to the input voltage, we can know that the rectiﬁer converts the voltage value to the power value. The output current of each rectiﬁers are added by simple current mirror, and R-C load is adopted to sense the DC value while attenuating the voltage ripple with the frequency of the input carrier signal.

 



B. LP-Mode Receiver The FM-radio band from  $80\ \mathrm{MH}z$  to  $110~\mathrm{MHz}$  becomes the strong interference to the HBC transceiver due to the body an- tenna effect. The direct conversion BPSK receiver consumes large amount of power to achieve good selectivity, but RSSI and the proposed LP-mode receiver can relax the selectivity overhead. Fig. 14 shows the concept and the architecture of the LP-mode receiver. When the input power we can know from the RSSI is over $^{-35}$  dBm which is  $10~\mathrm{dB}$  larger than maximum FM-radio input power, the LP-mode receiver which reuses the ampliﬁers in RSSI and digital demodulator can replace the di- rect conversion receiver. Since the power consumption of the direct conversion receiver becomes zero and the power con- sumption of the reused ampliﬁers are very low because of low required bandwidth  $(21\,\mathrm{MHz})$ , the LP-mode receiver can reduce the  $60\%$  of power consumption.

 



Fig. 15 shows the signal recovery ﬂow and demodulator circuit. Data from the human body is recovered to the digital data by ampliﬁer chain. In the demodulator, the input data is ﬁrstly sampled by  $42~\mathrm{MHz}$  clock. A Logic with several gates which is shown in Fig. 16 helps to ﬁnd the data non-inverting time to demodulate the BPSK data. The phase ambiguity can be solved in the PHY processor by comparing the input data with the promising data sequence of preamble stage in data packet.

# C. Reconﬁgurable LNA With RSSI

Fig. 16 shows the operation of the reconﬁgurable LNA. If the voltage gain of the LNA is ﬁxed, LNA should have both good sensitivity and good linearity to cover the both weak input and strong input, resulting in large power consumption in LNA. On the other hand, since we can know the input signal power by using the RSSI, the LNA can adaptively control the sensitivity and linearity performance with much lower power consumption. For instance, if the input signal is strong, the LNA is controlled to locate in (1) with the good linearity, and if the input signal is weak, the LNA is adjusted to locate in (3) with the good noise performance, which leads to reducing the power consumption by  $60\%$ .

VI. MEASUREMENT RESULTS Fig. 17 shows the TX spectrum measurement results. The spectrum with various data rate is measured and white line shows the required mask. All the measurement results satisfy the mask requirements. The spectrum of low frequency band and high frequency band cannot be shown in one screen due to the noise ﬂoor of the spectrum analyzer.

 



Fig. 18 shows the measured eye diagram of the direct con- version receiver and LP-mode receiver. The eye diagram of the direct conversion receiver is measured with the $-70~\mathrm{dBm}$  input power at 1.3125 Mbps data rate. The eye diagram, of the LP-mode receiver is measured with $-30$  dBm input power at 1.3125 Mbps.

Fig. 19 shows the measurement results of the transmitter output power. The output power is controlled by changing the MOS size of the transmitter driver. The maximum output power is around $-10~\mathrm{dBm}$ , and measured EVM of the BPSK modulated data from the TX driver is  $6.4\%$ . The error is mainly caused by analog ﬁlters.

Fig. 20(a) shows the blocker measurement results. Since the human body height is similar to the half of the FM-radio wave- length, the FM-radio signal can be easily coupled to the human body. The maximum input power of the FM-radio interference, or FM-blocker, is $^{-38}$  dBm. Fig. 20(b) shows the relationship between input FM-blocker power and receiver sensitivity. If the FM-blocker power is larger than $-35~\mathrm{dBm},$ , the BER becomes larger than  $10\%$ . The sensitivity becomes lower when the input FM-blocker becomes lower.

Fig. 21 shows the RSSI measurement results. The X axis is the input signal power, left Y axis is output RSSI output voltage and right Y axis is the error between input signal power and calculated input power from the RSSI output voltage. The mea- sured input dynamic range is from $-10$  dBm to $-100\:\mathrm{dBm}$  and the measured error is under 2 dBm in $-20$  dBm to $-80$  dBm input range.

Fig. 22 shows the chip photograph and performance summary table. The proposed HBC transceiver is implemented in SMIC  $130\ \mathrm{nm}$  process and it satisﬁes all the standard requirements including sensitivity, data-rate, modulation, spectral mask and

TABLE I PERFORMANCE COMPARISON TABLE

energy detection capability. The transmitter power consumption is  $1.3\;\mathrm{mW}$  and the power consumption of the direct conversion receiver at sample time and at hold time is  $6.2\,\mathrm{mW}$  and  $3.8\,\mathrm{mW},$  respectively. The RSSI successfully measure the input signal power with the $-100~\mathrm{dBm}$  to $-10~\mathrm{dBm}$  input dynamic range. The LP-mode receiver and reconﬁgurable-LNA utilize the RSSI output to reduce the power consumption, resulting in  $50\%$  and  $60\%$  power reduction, respectively.

Table I shows the comparison table with previous HBC trans- ceivers, The proposed transceiver in this work fully satisﬁes the HBC standard and lower power consumption, simultaneously, compared with the previous works.

VII. CONCLUSION In this paper, the IEEE 802.15.6 fully compatible HBC trans- ceiver is proposed with four low-power features. First, com- pared with the high speed DAC architecture, analog active ﬁlter based TX mask ﬁlter is adopted with  $30\%$  power reduction. Second, the DLL-based BPSK receiver saves the  $40\%$  power consumption by turning off the  $\mathsf Q$  data path with the S/H op- eration. Third, the LP-mode receiver with the measured input power by RSSI reduces the  $50\%$  power consumption. At last, the reconﬁgurable LNA with the RSSI output information is adopted with  $60\%$  power reduction. As a result, the proposed standard compatible receiver implemented in  $130~\mathrm{{nm}}$  CMOS technology consumes as low as  $5.2\;\mathrm{mW}$  power while satisfying all the standard requirements.



REFERENCES [1] T. Zimmerman, “Personal area network: Near-ﬁeld intra-body commu- nication,” IBM Syst. J., vol. 35, no. 3–4, pp. 609–617, 1996.



[2] N. Cho et al., “The human body characteristics as a signal transmission medium for intra-body communication,” IEEE Trans. Microw. Theory Techn., vol. 55, no. 5, pp. 1080–1086, May 2007.

[3] J. Bae et al., “The signal transmission mechanism on the surface of human body for body channel communication,” IEEE Trans. Microw. Theory Techn., vol. 60, no. 3, pp. 582–593, Mar. 2012.

[4] S. Song et al., “A  $0.9~\mathrm{V}~2.6~\mathrm{mW}$  Body-coupled scalable PHY trans- ceiver for body sensor applications,” in IEEE ISSCC Dig. Tech. Pa- pers, 2007, pp. 366–367.

[5] N. Cho et al., “A  $60\,\mathrm{kb/s}$ -t $\mathrm{o}\mathrm{-}10\,\mathrm{Mb/s}\,0.37\,\mathrm{nJ/b}$  adaptive-frequency-hop- ping transceiver for interference resilient body channel communica- tion,” IEEE J. Solid-State Circuits, vol. 44, no. 3, pp. 708–717, Mar. 2009.

[6] N. Cho et al., “A  $10.8\;\mathrm{mW}$  body channel communication/MICS dual- band transceiver for a uniﬁed body sensor network controller,” IEEE J. Solid-State Circuits, vol. 44, no. 12, pp. 3459–3468, Dec. 2009.

[7] A. Fazzi et al., “A  $2.75~\mathrm{mW}$  wideband correlation-based transceiver for body-coupled communication,” in IEEE ISSCC Dig. Tech. Papers, 2009, pp. 204–205.

[8] J. Bae et al., “A  $0.24\mathrm{{-nJ/b}}$  wireless body-area-network transceiver with scalable double-FSK modulation,” IEEE J. Solid-State Circuits, vol. 47, no. 1, pp. 310–322, Jan. 2012.

[9] H. Lee et al., “A  $5.5~\mathrm{mW}$  IEEE 802.15.6 wireless body area network standard transceiver for multi-channel electro-acupuncture applica- tion,” in IEEE ISSCC Dig. Tech. Papers, 2013, pp. 452–453.

[10] J. Lee et al., “A  $60\,\mathrm{Mb/s}$  wideband BCC transceiver with  $150\;\mathrm{pJ/b}$  RX and  $31~\mathrm{pJ/b}$  TX for emerging wearable applications,” in IEEE ISSCC Dig. Tech. Papers, 2014, pp. 498–499.

[11] J. Bae et al., “A low energy crystal-less double-FSK transceiver for wireless body-area-network,” in IEEE A-SSCC Dig. Tech. Papers, 2011, pp. 181–184.

[12] H. Lee et al., “A  $33\ \mu\mathrm{W}/\mathrm{\Omega}$ W/node duty cycle controlled HBC transceiver system for medical BAN with 64 sensor nodes,” in Proc. IEEE Custom Integrated Circuits Conf., 2014, pp. 1–8.

[13] H. Cho et al., “A  $5.2~\mathrm{mW}$  IEEE 802.15.6 HBC standard compatible transceiver with power efﬁcient delay-locked-loop based BPSK de- modulator,” in IEEE A-SSCC Dig. Tech. Papers, 2014, pp. 297–300.

[14] H. Cho et al., “A  $79~\mathrm{pJ/b}$  80 Mb/s full-duplex transceiver and a 42.5  $\mu\mathrm{W}\ 100\,\mathrm{kb}/\mathrm{s}$  super-regenerative transceiver for body channel commu- nications,” in IEEE ISSCC Dig. Tech. Papers, 2015, pp. 380–381.



[15] IEEE Standard for Local and Metropolitan Area Networks: Part 15.6 Wireless Body Area Networks, IEEE Computer Society, IEEE Stan- dards Association, Feb. 29, 2012.

[16] D. Kim et al., “A 622-Mb/s mixed-mode BPSK demodulator using a half-rate bang-bang phase detector,” IEEE J. Solid-State Circuits, vol. 43, no. 10, pp. 2284–2292, Oct. 2008.

[17] S. Huang et al., “W-band BPSK and QPSK transceiver with Costas- loop carrier recovery in 65-nm CMOS technology,” IEEE J. Solid-State Circuits, vol. 46, no. 12, pp. 3033–3046, Dec. 2011.

[18] M. Kitsunezuka et al., “A 30-MHz 2.4-GHz CMOS receiver with in- tegrated RF ﬁlter and dynamic-range-scalable energy detector for cog- nitive radio systems,” IEEE J. Solid-State Circuits, vol. 4, no. 5, pp. 1084–1093, May 2012.

[19] P. Huang et al., “A 2-V 10.7-MHz CMOS limiting ampliﬁer/RSSI,” IEEE J. Solid-State Circuits, vol. 35, no. 10, pp. 1474–1480, Oct. 2000. [20] C. Wu et al., “A 110-MHz 84-dB CMOS programmable gain ampliﬁer with integrated RSSI function,” IEEE J. Solid-State Circuits, vol. 40, no. 6, pp. 1249–1258, Jun. 2005.

Hyunwoo Cho (S'10) received the B.S. degree from the Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 2010, and the M.S. degree from the Department of Electrical Engineering, KAIST, in 2012, where he is currently working toward the Ph.D. degree.

He has worked on developing power and energy- efﬁcient CMOS wireless transceivers for portable and wearable devices working around the human body. His current research interest is low-power and low-energy body-area-network transceiver design and body-channel characteristics analysis.

Hyungwoo Lee (M'15) received the B.S., M.S., and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 2010, 2012, and 2015, respectively. His Ph.D. research concerned wireless body area network circuits and systems and is focused on the network optimization among multiple sensor nodes using the human body communication PHY&MAC.

Since 2015, he has been with Samsung Advanced Institute of Technology (SAIT), Samsung Elec- tronics. He is currently a research staff member and his research interest is in low-energy system design for mobile healthcare applications.

Joonsung Bae (S'07) graduated from the Electrical Engineering Department of Korea Advanced Insti- tute of Science and Technology (KAIST), Daejeon, Korea, in 2007 and received the M.S. and Ph.D. de- grees in electrical engineering from KAIST in 2009 and 2013, respectively. His Ph.D. work concerned the Wireless Body Area Network (WBAN) circuits and systems.

In 2012, he was a visiting scholar of IMEC, Bel- gium, and researched noise analysis of the dry elec- trode for body channel communication. Since 2013, he has been with the Memory Business of Samsung Electronics, Korea, where he developed the SoC Design for the SSD (Solid-State Drive) and UFS (Uni- versal Flash Storage). As a senior engineer, he designed the integrated circuits for PCI-Express 3.0 and M-PHY 3.0. In 2014, he joined the Information and Electronics Research Institute in KAIST as a postdoctoral researcher. Now he is an analog circuit designer in IMEC, Belgium, where he investigates ultra-low- power biomedical circuits. His current research interests are high-speed serial interface PHY, short-range wireless connections, WBAN circuits and systems, and biomedical circuits and systems.



Dr. Bae received the Asian Solid-State Circuits Conference (A-SSCC) Best Design Awards in 2011 and the Global Internship Scholarship of National Re- search Foundation of Korea in 2012.

Hoi-Jun Yoo (M'95–SM'04–F'08) graduated from the Electronic Department of Seoul National Uni- versity, Seoul, Korea, in 1983 and received the M.S. and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 1985 and 1988, respectively. Since 1998, he has been on the faculty of the Department of Electrical Engineering at KAIST and now is a full Professor. From 2001 to 2005, he was the Director of Korean System Integration and IP Authoring Research Center (SIPAC). From 2003 to 2005, he was the full time Advisor to Minister of Korea Ministry of Information and Communication and National Project Manager for SoC and Computer. In 2007, he founded System Design Innovation & Application Research Center (SDIA) at KAIST. Since 2010, he has served on the general chair of Korean Institute of Next Generation Computing. His current interests are computer vision SoC, body area networks, biomedical devices and circuits. He is a co-author of DRAM Design (Korea: Hongleung, 1996), High Performance DRAM (Korea: Sigma, 1999), Networks on Chips (Morgan Kaufmann, 2006), Low-Power NoC for High-Performance SoC Design (CRC Press, 2008), Circuits at the Nanoscale (CRC Press, 2009), Embedded Memories for Nano-Scale VLSIs (Springer, 2009), Mobile 3D Graphics SoC from Algorithm to Chip (Wiley, 2010), and Bio-Medical CMOS ICs (Springer, 2011).

Dr. Yoo received the Electronic Industrial Association of Korea Award for his contribution to DRAM technology in 1994, Hynix Development Award in 1995, the Korea Semiconductor Industry Association Award in 2002, Best Re- search of KAIST Award in 2007, Scientist/Engineer of this month Award from Ministry of Education, Science and Technology of Korea in 2010, Best Schol- arship Awards of KAIST in 2011, and Order of Service Merit from Ministry of Public Administration and Security of Korea in 2011 and has been co-recipients of ASP-DAC Design Award 2001, Outstanding Design Awards of 2005, 2006, 2007, 2010, 2011 A-SSCC, Student Design Contest Award of 2007, 2008, 2010, 2011 DAC/ISSCC. He has served as a member of the executive committee of ISSCC, Symposium on VLSI, and A-SSCC and the TPC chair of the A-SSCC 2008 and ISWC 2010, IEEE Fellow, IEEE Distinguished Lecturer ('10–'11), Far East Chair of ISSCC ('11–'12), Technology Direction Sub-Committee Chair of ISSCC ('13), TPC Vice Chair of ISSCC ('14), and TPC Chair of ISSCC ('15).
A 5.2 mW IEEE 802.15.6 HBC Standard Compatible Transceiver With Power Efﬁcient Delay-Locked-Loop Based BPSK Demodulator

Hyunwoo Cho, Student Member, IEEE, Hyungwoo Lee, Student Member, IEEE, Joonsung Bae, Member, IEEE, and Hoi-Jun Yoo, Fellow, IEEE

Abstract—A Low-power IEEE 802.15.6 human body communi- cation (HBC) fully compatible transceiver is implemented in 130 nm CMOS process. In this work, the proposed HBC transceiver satisfying all the standard requirements has four key features for low power consumption which includes: 1) low-power analog ac- tive ﬁlters for TX spectral mask:  $\mathbf{30\%}$  power reduction; 2) de- layed locked loop (DLL) based BPSK receiver with the S/H oper- ation for turning off unnecessary blocks:  $40\%$  power reduction; 3) low-power mode (LP-mode) receiver with the received signal strength indicator (RSSI) output data:  $\mathbf{50\%}$  power reduction; and 4) reconﬁgurable LNA with RSSI output data:  $\mathbf{60\%}$  power reduc- tion. As a result, the proposed transceiver can fully satisfy the HBC standard requirements while consuming  $\pmb{5.2}\ \mathbf{mW}$  from the  $\mathbf{1.2\V}$  supply.

Index Terms—Costas loop, delayed-locked loop (DLL), human body communication (HBC), IEEE 802.15.6, received signal strength indicator (RSSI), spectral mask, standard, wireless body area network (WBAN).

I. INTRODUCTION R ECENTLY, wireless body area network (WBAN) is getting more and more attention in the emerging appli- cations which combines healthcare and consumer electronics around the human body. The major design challenge associated with the WBAN is to extend the lifetime of the WBAN devices under limited energy source. Human body communication (HBC) which uses the human body as a communication channel is considered as a power-efﬁcient wireless communication solution of the WBAN because high conductivity of the human body in a low frequency band enables low power communica- tion. Since the HBC was ﬁrstly presented in [1], the physical communication principle and a variety of power efﬁcient HBC transceivers have been presented [2]–[14], and eventually in 2012 the WBAN is standardized as IEEE 802.15.6 [15]. However, fully satisfying all requirements of HBC standard (Fig. 1) while consuming low power for long battery life is very difﬁcult because of 1) transmitter (TX) mask with low spectrum ﬂoor, sharp slope and wide frequency bandwidth, 2) low sensitivity, and 3) received energy detection capability for PHY/MAC protocol.



The previous HBC transceiver was proposed in 2013 [9], but it was not optimized for low power consumption. In the TX, the power-consuming high-order digital ﬁlter which consists of high speed DAC with high speed clock generation  $(>\!840\,\mathrm{MHz})$  to avoid aliasing effect was adopted to satisfy the mask require- ments. In the RX, the power-hungry VCO-based BPSK demod- ulator with I (in phase) and Q (quadrature phase) dual data paths for coherent receiving was employed in RX, which requires sig- niﬁcant power consumption. In addition, the energy detector was not implemented which is an essential block for PHY/MAC protocol.

In this paper, we propose a low power and fully IEEE 802.  $15.6\,\mathrm{HBC}$  compatible transceiver satisfying all the tight speciﬁ- cations by adopting 4 key techniques. First, in the TX, an analog active ﬁlter rather than the high speed DAC reduces the power consumption by  $30\%$  because bandwidth requirement of analog active ﬁlter is much lower than the DAC in [9]. Second, the delay-locked-loop (DLL) based BPSK demodulator mitigates  $40\%$  of power consumption by turning off synchronization cir- cuits in Q-path data path with the help of sample and hold (S/H) circuit which retains the control voltage of the DLL. Third, the received signal strength indicator (RSSI), which measures the received signal power to obtain the energy detection ca- pability for the energy-efﬁcient PHY/MAC operation, enables the operation of the low-power mode (LP-mode) receiver com- posed of ampliﬁers used in RSSI and digital demodulator while turning off all blocks of the DLL-based BPSK receiver with  $50\%$  power reduction. At last, by utilizing the information of the input signal power, the noise and linearity performance of the LNA can be adaptively controlled with the  $60\%$  reduced bias current consumption.

The rest of this paper is organized as follows. In the Section II, the overall architecture of the proposed HBC transceiver will be introduced. In the Section III and IV, the detail design of the transmitter with the analog active ﬁlter and DLL-based BPSK receiver will be explained, respectively. In Section V, the low power schemes including LP-mode receiver and reconﬁgurable LNA will be presented with the RSSI design. After illustrating the measurement results in Section VI, the conclusion will be summarized in Section VII.

Fig. 2 shows the overall transceiver architecture which is composed of a transmitter with a spectral mask ﬁlter, a direct conversion receiver with a BPSK synchronizer, a RSSI and a digital baseband (PHY and MAC). In the transmitter, the base- band data from the PHY/MAC is modulated with 21 MHz car- rier frequency. The modulation is the frequency selective dig- ital transmission (FSDT) which is the same with the rectangular wave modulated BPSK. The spectral mask ﬁlter suppresses the low frequency band and high frequency band of the modulated data, and the driver transmits the data through the human body. In the receiver, the direct conversion receiver with the BPSK synchronizer recovers the received data from the LNA. The dif- ferential input LNA is adopted to generate differential output without noise ﬁgure growth which is essential for the direct conversion receiver conﬁguration. Since the TX is disconnected when the receiver is operated, it does not generate LNA input mismatch. I and Q data paths of the direct conversion receiver consists of active mixers, baseband ampliﬁers, baseband low- pass ﬁlters, and limiting ampliﬁers. The DLL-based BPSK syn- chronizer adjusts its local RX clock signal to the rising and falling edge of the input data. Especially compared with the VCO-based architecture [15], [16], the DLL-based architecture can promptly synchronize its clock with the input data without any stability problem. Moreover, the proposed synchronizer in Q data path including mixer, baseband ampliﬁer, low pass ﬁlter and limiting ampliﬁer is disabled to signiﬁcantly decrease its power consumption when the phase synchronization loop is sta- bilized. The RSSI is composed of a seven stage limiting ampli- ﬁer with seven rectiﬁers and 8-bit single-slope ADC. The RSSI measures the received input power with a range of $-100$  dBm to  $-10\:\mathrm{dBm}$ . Based on the output of the RSSI, the operation mode of the LNA is controlled to optimize its consuming power while satisfying the sensitivity requirements, resulting in  $60\%$  power

 



CHO et al.

reduction. In addition, to further reduce the power consumption when the input power is larger than $-35~\mathrm{dBm}$ , the direct con- version receiver circuits are turned off, and only the LP-mode receiver with the limiting ampliﬁer and digital demodulator for BPSK is turned on to recover the data. In the LP-mode receiver, over  $50\%$  of power consumption can be reduced.

quired to remove the harmonic frequency and alias terms, which leads to the signiﬁcant power consumption.

Fig. 3(b) shows the proposed transmitter architecture [13] with the low-power consuming analog active ﬁlters. Compared with the high speed DAC and digital ﬁlter, the analog active ﬁlter consumes much lower power because the OTA bandwidth requirement in the analog ﬁlter is much lower than the DAC. Since the spectral mask is falling at the  $23.625\,\mathrm{MHz}$  and the end frequency is  $400\,\mathrm{MHz},$ , the 3 dB frequency and gain bandwidth should be large than  $23.625~\mathrm{MHz}$  and  $400~\mathrm{MHz},$ , respectively. In this work, we designed the OTA 3 dB bandwidth as  $100\,\mathrm{MHz}$  which is larger than the  $23.625\;\mathrm{MHz}$ . Compared with the DAC which requires at least  $800\,\mathrm{MHz}$  sampling frequency, the analog ﬁlter requires only  $100\,\mathrm{MHz}$  bandwidth, and this bandwidth re- duction leads to  $30\%$  power reduction in the transmitter design. Although the proposed analog active ﬁlters require off-chip dig- itally controlled tuning to compensate the frequency character- istics variation caused by the process variation, it takes small cost because the tuning is required only at the ﬁrst time and the process variation can be avoided by using post layout sim- ulation and common centroid layout technique. The proposed transmitter is composed of the level shifter, analog active ﬁl- ters, driver and off-chip analog ﬁlters. The signal ﬂow in each stage in the transmitter will be explained in Fig. 4. Since the data is modulated with the rectangular wave swinging from  $0\mathrm{~V~}$  to  $1.2\;\mathrm{V},$  there are excessive DC power and harmonics in high fre- quency. Second, the data after the level shifter which reduces the swing range in order to avoid the non-linear effect in the following analog circuits is attenuated by 6th-order low-pass ﬁlter and 9th-order high-pass ﬁlter (Fig. 4(b)). The excessive spectrum power in DC and high frequency band is suppressed under the mask, but the non-linear characteristics of the analog active ﬁlter and driver increases the sideband spectrum, which is called spectrum regrowth effect (Fig. 4(c)). The amount of

 



CHO et al.

Fig. 9. Active mixer and its replica bias circuit.

Fig. 10. Delay circuit with the linear bias control circuit.

this increased low frequency spectrum power cannot be exactly predicted due to the PVT variation, so we adopt the simple R-C off-chip 2nd-order high-pass ﬁlter to additionally attenuate the low frequency power. After the off-chip 2nd-order high-pass ﬁlter, all the excessive spectrum power is removed, and ﬁnally, the output spectrum can satisfy the spectral mask requirement (Fig. 4(d)).

Fig. 5 shows the circuit of the 3rd-order active low-pass ﬁlter and high pass ﬁlter. The active ﬁlter adopts the multiple feed- back architecture which can obtain the $-60$  dB/decade slope with only one OTA. The OTA has folded cascade conﬁguration with the frequency compensation technique [14].

IV. RECEIVER DESIGN

# A. DLL-Based BPSK Receiver

Fig. 6 shows the analysis of the DLL-based BPSK receiver. In the BPSK receiver, the local clock synchronization path, which is called Costas loop, is essential. The receiver should have I path and Q path to calculate the phase difference between the input data and local clock by multiplying output of both paths. Since the multiplied output of I and Q path is proportional to , the phase difference can be zero by composing the syn- chronization loop to make thezero. In the conventional works [16], [17], the Costas loop uses the VCO to calibrate the input signal and local clock, which is quite similar with the PLL. Such implementations, however, may have stability concerns. In this work, we proposed a DLL-based Costas loop, which avoid the stability issues associated with 2nd (or higher order) phase-locked loops. Since the frequency of local clock is not changed with respect to the control voltage, the delay cell control voltage can be held with the help of sample and hold buffer while turning off the  $\mathsf Q$  path blocks, reducing  $40\%$  of power consumption. The phase error due to the  $\mathrm{S/H}$  buffer offset ( $^{<2}$  degrees) is negligible. In addition, the clock difference be- tween the TX and RX can be calibrated in the packet domain. In the HBC standard packet, the pilot which is composed of

 



Fig. 16. Reconﬁgurable LNA operation.

simple 0,1,0,1 for clock calibration must be inserted if the pay- load length is longer than 64 bytes. Since the clock drift of the crystal oscillator is usually under 10 ppm and it must be smaller than  $100\;\mathrm{ppm}$  in HBC standard, the clock offset can be period- ically calibrated at the pilot time without BER degradation.

Fig. 7 shows the detailed block diagram of the proposed DLL-based BPSK receiver. I path and Q path are composed of mixer buffer, mixer, baseband ﬁlter, and limiting ampliﬁer. The voltage to current converter and a capacitor make the 1st-order loop ﬁlter, and S/H buffer with a switch is employed for holding the control voltage while turning off the Q path blocks. The delayed 21 MHz reference clock signal through the delay cell in Costas loop works as a local clock of the RX.

Fig. 19. TX output power measurement results.

B. Circuit Design Fig. 8 shows the schematic of the reconﬁgurable LNA [8]. The common gate LNA with the-boosting cross coupled NMOS is adopted for low noise characteristics. The size of the input NMOSandare controlled by 6 bit control code to adaptively change the LNA performance for low-power con- sumption which will be explained in Section V, and the noise ﬁgure and the 1 dB compression point value in accordance with the control code is shown in Fig. 8.



Fig. 9 shows the schematic of the active mixer and replica bias circuit. The active mixer has a simple Gilbert mixer conﬁgura- tion, and resistoris added to reducing the switching noise and increasing the linearity of mixer. The resistorand are composing the replica bias circuit. It has a same conﬁg- uration of the half circuit of the Gilbert mixer, but resistor value is four times larger and size of transistor is four times smaller than the original ones, reducing the current consumption of the replica bias circuit by  $25\%$ .



Fig. 10 presents the delay cell and delay control bias cir- cuit. The delay cell is composed of six stages inverter based unit delay cells whose bias current is supplied by the delay con- trol bias circuit. In the bias circuit, the output current is linearly changed by the input control voltage. The two differen- tial MOS pairsremove the 2nd-order harmonics to increase the linearity while consuming more power consump- tion. Fig. 11 shows the measurement results of the delay time of the delay cell in accordance with the control voltageand bias current.

V. RSSI AND LOW-POWER RECEIVER

# A. RSSI Design

In the PHY/MAC protocol for multi-nodes operation, the en- ergy detection block is an essential. Fig. 12 shows the principle and architecture of the RSSI for energy detection [18]–[20]. The RSSI is composed of seven ampliﬁers, seven rectiﬁers, current adder with low-pass ﬁlter and ADC. Since the number of the ampliﬁers decides the dynamic range of the RSSI and we need  $-10$  dBm to $-100\:\mathrm{dBm}$  input dynamic range to sense the HBC input signal power, seven ampliﬁers with the voltage gain of 15 dB and seven rectiﬁers are adopted. Each ampliﬁer and rectiﬁer covers the different input power range, and we can achieve wide dynamic range by adding all rectiﬁers' results.

 



Fig. 13 shows the schematic of the rectiﬁer and current adder. The output current of each of the rectiﬁer with the size ratio of k can be expressed as (1) Since the output current is proportional to the input voltage, we can know that the rectiﬁer converts the voltage value to the power value. The output current of each rectiﬁers are added by simple current mirror, and R-C load is adopted to sense the DC value while attenuating the voltage ripple with the frequency of the input carrier signal.

 



B. LP-Mode Receiver The FM-radio band from  $80\ \mathrm{MH}z$  to  $110~\mathrm{MHz}$  becomes the strong interference to the HBC transceiver due to the body an- tenna effect. The direct conversion BPSK receiver consumes large amount of power to achieve good selectivity, but RSSI and the proposed LP-mode receiver can relax the selectivity overhead. Fig. 14 shows the concept and the architecture of the LP-mode receiver. When the input power we can know from the RSSI is over $^{-35}$  dBm which is  $10~\mathrm{dB}$  larger than maximum FM-radio input power, the LP-mode receiver which reuses the ampliﬁers in RSSI and digital demodulator can replace the di- rect conversion receiver. Since the power consumption of the direct conversion receiver becomes zero and the power con- sumption of the reused ampliﬁers are very low because of low required bandwidth  $(21\,\mathrm{MHz})$ , the LP-mode receiver can reduce the  $60\%$  of power consumption.

 



Fig. 15 shows the signal recovery ﬂow and demodulator circuit. Data from the human body is recovered to the digital data by ampliﬁer chain. In the demodulator, the input data is ﬁrstly sampled by  $42~\mathrm{MHz}$  clock. A Logic with several gates which is shown in Fig. 16 helps to ﬁnd the data non-inverting time to demodulate the BPSK data. The phase ambiguity can be solved in the PHY processor by comparing the input data with the promising data sequence of preamble stage in data packet.

# C. Reconﬁgurable LNA With RSSI

Fig. 16 shows the operation of the reconﬁgurable LNA. If the voltage gain of the LNA is ﬁxed, LNA should have both good sensitivity and good linearity to cover the both weak input and strong input, resulting in large power consumption in LNA. On the other hand, since we can know the input signal power by using the RSSI, the LNA can adaptively control the sensitivity and linearity performance with much lower power consumption. For instance, if the input signal is strong, the LNA is controlled to locate in (1) with the good linearity, and if the input signal is weak, the LNA is adjusted to locate in (3) with the good noise performance, which leads to reducing the power consumption by  $60\%$ .

VI. MEASUREMENT RESULTS Fig. 17 shows the TX spectrum measurement results. The spectrum with various data rate is measured and white line shows the required mask. All the measurement results satisfy the mask requirements. The spectrum of low frequency band and high frequency band cannot be shown in one screen due to the noise ﬂoor of the spectrum analyzer.

 



Fig. 18 shows the measured eye diagram of the direct con- version receiver and LP-mode receiver. The eye diagram of the direct conversion receiver is measured with the $-70~\mathrm{dBm}$  input power at 1.3125 Mbps data rate. The eye diagram, of the LP-mode receiver is measured with $-30$  dBm input power at 1.3125 Mbps.

Fig. 19 shows the measurement results of the transmitter output power. The output power is controlled by changing the MOS size of the transmitter driver. The maximum output power is around $-10~\mathrm{dBm}$ , and measured EVM of the BPSK modulated data from the TX driver is  $6.4\%$ . The error is mainly caused by analog ﬁlters.

Fig. 20(a) shows the blocker measurement results. Since the human body height is similar to the half of the FM-radio wave- length, the FM-radio signal can be easily coupled to the human body. The maximum input power of the FM-radio interference, or FM-blocker, is $^{-38}$  dBm. Fig. 20(b) shows the relationship between input FM-blocker power and receiver sensitivity. If the FM-blocker power is larger than $-35~\mathrm{dBm},$ , the BER becomes larger than  $10\%$ . The sensitivity becomes lower when the input FM-blocker becomes lower.

Fig. 21 shows the RSSI measurement results. The X axis is the input signal power, left Y axis is output RSSI output voltage and right Y axis is the error between input signal power and calculated input power from the RSSI output voltage. The mea- sured input dynamic range is from $-10$  dBm to $-100\:\mathrm{dBm}$  and the measured error is under 2 dBm in $-20$  dBm to $-80$  dBm input range.

Fig. 22 shows the chip photograph and performance summary table. The proposed HBC transceiver is implemented in SMIC  $130\ \mathrm{nm}$  process and it satisﬁes all the standard requirements including sensitivity, data-rate, modulation, spectral mask and

TABLE I PERFORMANCE COMPARISON TABLE

energy detection capability. The transmitter power consumption is  $1.3\;\mathrm{mW}$  and the power consumption of the direct conversion receiver at sample time and at hold time is  $6.2\,\mathrm{mW}$  and  $3.8\,\mathrm{mW},$  respectively. The RSSI successfully measure the input signal power with the $-100~\mathrm{dBm}$  to $-10~\mathrm{dBm}$  input dynamic range. The LP-mode receiver and reconﬁgurable-LNA utilize the RSSI output to reduce the power consumption, resulting in  $50\%$  and  $60\%$  power reduction, respectively.

Table I shows the comparison table with previous HBC trans- ceivers, The proposed transceiver in this work fully satisﬁes the HBC standard and lower power consumption, simultaneously, compared with the previous works.

VII. CONCLUSION In this paper, the IEEE 802.15.6 fully compatible HBC trans- ceiver is proposed with four low-power features. First, com- pared with the high speed DAC architecture, analog active ﬁlter based TX mask ﬁlter is adopted with  $30\%$  power reduction. Second, the DLL-based BPSK receiver saves the  $40\%$  power consumption by turning off the  $\mathsf Q$  data path with the S/H op- eration. Third, the LP-mode receiver with the measured input power by RSSI reduces the  $50\%$  power consumption. At last, the reconﬁgurable LNA with the RSSI output information is adopted with  $60\%$  power reduction. As a result, the proposed standard compatible receiver implemented in  $130~\mathrm{{nm}}$  CMOS technology consumes as low as  $5.2\;\mathrm{mW}$  power while satisfying all the standard requirements.



REFERENCES [1] T. Zimmerman, “Personal area network: Near-ﬁeld intra-body commu- nication,” IBM Syst. J., vol. 35, no. 3–4, pp. 609–617, 1996.



[2] N. Cho et al., “The human body characteristics as a signal transmission medium for intra-body communication,” IEEE Trans. Microw. Theory Techn., vol. 55, no. 5, pp. 1080–1086, May 2007.

[3] J. Bae et al., “The signal transmission mechanism on the surface of human body for body channel communication,” IEEE Trans. Microw. Theory Techn., vol. 60, no. 3, pp. 582–593, Mar. 2012.

[4] S. Song et al., “A  $0.9~\mathrm{V}~2.6~\mathrm{mW}$  Body-coupled scalable PHY trans- ceiver for body sensor applications,” in IEEE ISSCC Dig. Tech. Pa- pers, 2007, pp. 366–367.

[5] N. Cho et al., “A  $60\,\mathrm{kb/s}$ -t $\mathrm{o}\mathrm{-}10\,\mathrm{Mb/s}\,0.37\,\mathrm{nJ/b}$  adaptive-frequency-hop- ping transceiver for interference resilient body channel communica- tion,” IEEE J. Solid-State Circuits, vol. 44, no. 3, pp. 708–717, Mar. 2009.

[6] N. Cho et al., “A  $10.8\;\mathrm{mW}$  body channel communication/MICS dual- band transceiver for a uniﬁed body sensor network controller,” IEEE J. Solid-State Circuits, vol. 44, no. 12, pp. 3459–3468, Dec. 2009.

[7] A. Fazzi et al., “A  $2.75~\mathrm{mW}$  wideband correlation-based transceiver for body-coupled communication,” in IEEE ISSCC Dig. Tech. Papers, 2009, pp. 204–205.

[8] J. Bae et al., “A  $0.24\mathrm{{-nJ/b}}$  wireless body-area-network transceiver with scalable double-FSK modulation,” IEEE J. Solid-State Circuits, vol. 47, no. 1, pp. 310–322, Jan. 2012.

[9] H. Lee et al., “A  $5.5~\mathrm{mW}$  IEEE 802.15.6 wireless body area network standard transceiver for multi-channel electro-acupuncture applica- tion,” in IEEE ISSCC Dig. Tech. Papers, 2013, pp. 452–453.

[10] J. Lee et al., “A  $60\,\mathrm{Mb/s}$  wideband BCC transceiver with  $150\;\mathrm{pJ/b}$  RX and  $31~\mathrm{pJ/b}$  TX for emerging wearable applications,” in IEEE ISSCC Dig. Tech. Papers, 2014, pp. 498–499.

[11] J. Bae et al., “A low energy crystal-less double-FSK transceiver for wireless body-area-network,” in IEEE A-SSCC Dig. Tech. Papers, 2011, pp. 181–184.

[12] H. Lee et al., “A  $33\ \mu\mathrm{W}/\mathrm{\Omega}$ W/node duty cycle controlled HBC transceiver system for medical BAN with 64 sensor nodes,” in Proc. IEEE Custom Integrated Circuits Conf., 2014, pp. 1–8.

[13] H. Cho et al., “A  $5.2~\mathrm{mW}$  IEEE 802.15.6 HBC standard compatible transceiver with power efﬁcient delay-locked-loop based BPSK de- modulator,” in IEEE A-SSCC Dig. Tech. Papers, 2014, pp. 297–300.

[14] H. Cho et al., “A  $79~\mathrm{pJ/b}$  80 Mb/s full-duplex transceiver and a 42.5  $\mu\mathrm{W}\ 100\,\mathrm{kb}/\mathrm{s}$  super-regenerative transceiver for body channel commu- nications,” in IEEE ISSCC Dig. Tech. Papers, 2015, pp. 380–381.



[15] IEEE Standard for Local and Metropolitan Area Networks: Part 15.6 Wireless Body Area Networks, IEEE Computer Society, IEEE Stan- dards Association, Feb. 29, 2012.

[16] D. Kim et al., “A 622-Mb/s mixed-mode BPSK demodulator using a half-rate bang-bang phase detector,” IEEE J. Solid-State Circuits, vol. 43, no. 10, pp. 2284–2292, Oct. 2008.

[17] S. Huang et al., “W-band BPSK and QPSK transceiver with Costas- loop carrier recovery in 65-nm CMOS technology,” IEEE J. Solid-State Circuits, vol. 46, no. 12, pp. 3033–3046, Dec. 2011.

[18] M. Kitsunezuka et al., “A 30-MHz 2.4-GHz CMOS receiver with in- tegrated RF ﬁlter and dynamic-range-scalable energy detector for cog- nitive radio systems,” IEEE J. Solid-State Circuits, vol. 4, no. 5, pp. 1084–1093, May 2012.

[19] P. Huang et al., “A 2-V 10.7-MHz CMOS limiting ampliﬁer/RSSI,” IEEE J. Solid-State Circuits, vol. 35, no. 10, pp. 1474–1480, Oct. 2000. [20] C. Wu et al., “A 110-MHz 84-dB CMOS programmable gain ampliﬁer with integrated RSSI function,” IEEE J. Solid-State Circuits, vol. 40, no. 6, pp. 1249–1258, Jun. 2005.

Hyunwoo Cho (S'10) received the B.S. degree from the Department of Physics, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 2010, and the M.S. degree from the Department of Electrical Engineering, KAIST, in 2012, where he is currently working toward the Ph.D. degree.

He has worked on developing power and energy- efﬁcient CMOS wireless transceivers for portable and wearable devices working around the human body. His current research interest is low-power and low-energy body-area-network transceiver design and body-channel characteristics analysis.

Hyungwoo Lee (M'15) received the B.S., M.S., and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 2010, 2012, and 2015, respectively. His Ph.D. research concerned wireless body area network circuits and systems and is focused on the network optimization among multiple sensor nodes using the human body communication PHY&MAC.

Since 2015, he has been with Samsung Advanced Institute of Technology (SAIT), Samsung Elec- tronics. He is currently a research staff member and his research interest is in low-energy system design for mobile healthcare applications.

Joonsung Bae (S'07) graduated from the Electrical Engineering Department of Korea Advanced Insti- tute of Science and Technology (KAIST), Daejeon, Korea, in 2007 and received the M.S. and Ph.D. de- grees in electrical engineering from KAIST in 2009 and 2013, respectively. His Ph.D. work concerned the Wireless Body Area Network (WBAN) circuits and systems.

In 2012, he was a visiting scholar of IMEC, Bel- gium, and researched noise analysis of the dry elec- trode for body channel communication. Since 2013, he has been with the Memory Business of Samsung Electronics, Korea, where he developed the SoC Design for the SSD (Solid-State Drive) and UFS (Uni- versal Flash Storage). As a senior engineer, he designed the integrated circuits for PCI-Express 3.0 and M-PHY 3.0. In 2014, he joined the Information and Electronics Research Institute in KAIST as a postdoctoral researcher. Now he is an analog circuit designer in IMEC, Belgium, where he investigates ultra-low- power biomedical circuits. His current research interests are high-speed serial interface PHY, short-range wireless connections, WBAN circuits and systems, and biomedical circuits and systems.



Dr. Bae received the Asian Solid-State Circuits Conference (A-SSCC) Best Design Awards in 2011 and the Global Internship Scholarship of National Re- search Foundation of Korea in 2012.

Hoi-Jun Yoo (M'95–SM'04–F'08) graduated from the Electronic Department of Seoul National Uni- versity, Seoul, Korea, in 1983 and received the M.S. and Ph.D. degrees in electrical engineering from the Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 1985 and 1988, respectively. Since 1998, he has been on the faculty of the Department of Electrical Engineering at KAIST and now is a full Professor. From 2001 to 2005, he was the Director of Korean System Integration and IP Authoring Research Center (SIPAC). From 2003 to 2005, he was the full time Advisor to Minister of Korea Ministry of Information and Communication and National Project Manager for SoC and Computer. In 2007, he founded System Design Innovation & Application Research Center (SDIA) at KAIST. Since 2010, he has served on the general chair of Korean Institute of Next Generation Computing. His current interests are computer vision SoC, body area networks, biomedical devices and circuits. He is a co-author of DRAM Design (Korea: Hongleung, 1996), High Performance DRAM (Korea: Sigma, 1999), Networks on Chips (Morgan Kaufmann, 2006), Low-Power NoC for High-Performance SoC Design (CRC Press, 2008), Circuits at the Nanoscale (CRC Press, 2009), Embedded Memories for Nano-Scale VLSIs (Springer, 2009), Mobile 3D Graphics SoC from Algorithm to Chip (Wiley, 2010), and Bio-Medical CMOS ICs (Springer, 2011).

Dr. Yoo received the Electronic Industrial Association of Korea Award for his contribution to DRAM technology in 1994, Hynix Development Award in 1995, the Korea Semiconductor Industry Association Award in 2002, Best Re- search of KAIST Award in 2007, Scientist/Engineer of this month Award from Ministry of Education, Science and Technology of Korea in 2010, Best Schol- arship Awards of KAIST in 2011, and Order of Service Merit from Ministry of Public Administration and Security of Korea in 2011 and has been co-recipients of ASP-DAC Design Award 2001, Outstanding Design Awards of 2005, 2006, 2007, 2010, 2011 A-SSCC, Student Design Contest Award of 2007, 2008, 2010, 2011 DAC/ISSCC. He has served as a member of the executive committee of ISSCC, Symposium on VLSI, and A-SSCC and the TPC chair of the A-SSCC 2008 and ISWC 2010, IEEE Fellow, IEEE Distinguished Lecturer ('10–'11), Far East Chair of ISSCC ('11–'12), Technology Direction Sub-Committee Chair of ISSCC ('13), TPC Vice Chair of ISSCC ('14), and TPC Chair of ISSCC ('15).
# Clinical Study Clinical Characteristics of Cerebral Venous Sinus Thrombosis in Patients with Systemic Lupus Erythematosus: A Single-Centre Experience in China

Li Wang, Hua Chen, Yao Zhang, Wanli Liu, Wenjie Zheng, Xuan Zhang, and Fengchun Zhang



Key Laboratory of Rheumatology and Clinical Immunology, Department of Rheumatology and Clinical Immunology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Ministry of Education, No. 1 Shuaifuyuan, Beijing 100730, China Department of Neurology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, No. 1 Shuaifuyuan, Beijing 100730, China MOE Key Laboratory of Protein Science, School of Life Sciences, Tsinghua University, Beijing 100084, China Correspondence should be addressed to Wenjie Zheng; wenjzheng@gmail.com Received 22 August 2014; Accepted 17 November 2014 Academic Editor: Guixiu Shi Copyright  $^\copyright$  2015 Li Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

   

  

 



Clinical characteristics of systemic lupus erythematosus (SLE) patients complicated with cerebral venous sinus thrombosis (CVST) between 2000 and 2013 were analyzed through this retrospective, single-centre study. Of 4747 hospitalized SLE patients, 17  $(0.36\%,$ , 12 females, average age 30) had CVST. Headache  $(88.2\%)$  was the most common neurological symptom followed by nausea or vomiting  $(47.1\%)$ , conscious disturbance  $(41.2\%)$ , edema of eyelids or conjunctiva  $(35.3\%)$ , blurred vision or diplopia  $(35.3\%)$ , and seizure  $(35.3\%)$ . Increased intracranial pressure (ICP) occurred in 13 cases  $(76.5\%)$ . Magnetic resonance venography (MRV) detected thrombosis in the transverse  $(82.4\%)$ , sigmoid  $(52.9\%)$ , and sagittal  $(35.3\%)$  sinuses, with frequent  $(70.6\%)$  multiple sinus occlusions. Compared to SLE patients without CVST, SLE patients with CVST had a higher prevalence of thrombocytopenia and positive antiphospholipid antibodies and a higher SLE disease activity index (SLEDAI) score. 13 patients achieved improvement following glucocorticoids and immunosuppressants treatment, as well as anticoagulants, while 3 patients died at the hospital. CVST is relatively rare in SLE and tends to occur in active lupus patients. Intracranial hypertension is common but nonspecific clinical feature, so MRV evaluation is necessary to establish a diagnosis. Aggressive treatment for the rapid control of SLE activity combined with anticoagulants can improve the prognosis.

## 1. Introduction

Systemic lupus erythematosus (SLE) is a heterogeneous, mul- tisystem autoimmune disease associated with the production of antibodies against self-antigens. Its pathologic basis is vasculitis induced by immune complex, which suggests that blood vessels might be an important target of autoimmunity overactivity. As a result, there is a wide variation in ves- sel involvement in SLE, including arteriostenosis and deep venous thrombosis, while glomerulonephritis and defused alveolar hemorrhage can also occur to a certain extent. Cerebral venous sinus thrombosis (CVST) is an uncommon disorder in SLE that is a manifestation of overlapping neu- ropsychiatric lupus (NPLE) [1] and vessel disease. CVST has been rarely reported and is potentially fatal to the patient due to the acute neurological deterioration that can occur. As such, practitioners should be familiar with this condition, so the correct diagnosis can be made as soon as possible to improve patient prognosis.



The etiology of CVST is complicated [2]. In SLE, the main culprits are vascular injuries caused by vasculitis. Antiphos- pholipid antibodies (APLs) and prothrombotic tendencies might also be responsible for CVST [3].

By investigating the clinical features of CVST in SLE and analyzing the differences between SLE patients with and without CVST, this study seeks to explore the characteristics and risk factors for CVST in SLE patients.

## 2. Patients and Methods

2.1. Patients. 4747 SLE patients admitted to the Peking Union Medical College Hospital from January 2000 to December 2013 were reviewed. All patients were diagnosed as having SLE according to 1997 American College of Rheumatology (ACR) revised classification criteria [4]. In the SLE cohort, after exclusion of patients with infectious and traumatic CVST or incomplete data, we identified 17 cases  $(0.36\%)$  complicated with CVST as indicated by clinical features, lumbar puncture, imaging materials (magnetic resonance venography (MRV)), and at least one neurologist’s confir- mation. For each $(n~=~51)$  $\begin{array}{r l r}{(n}&{{}=}&{17)}\end{array}$ =17), three age- and sex- matched controls (𝑛 = 51) were randomly selected from the contemporaneously admitted SLE patients without CVST. The local institutional review board approved the study. Because the study was based on a review of medical records that had been obtained for clinical purposes, the requirement for written informed consent was waived.

2.2. Statistical Analyses. SPSS version 16.0 (SPSS Inc., Chic- ago, USA) was used to statistically analyze the data. Numer- ical data and categorical data were expressed as mean  $\pm\ \mathrm{SD}$  (range) and percentage, respectively. The significance was estimated by Student’s 𝑡-test, Pearson’s chi-sq $^{<5}$ re, or Fishe $<$  exact test (when expected frequencies were <5). 𝑃 values < 0.05 were considered to be statistically significant.

## 3. Results

3.1. Demographic Characteristics. Among 17 SLE patients with CVST, 12  $(70.6\%)$  were women. The median age was 30 years old (range 12–52) and the SLE disease duration was 12 months (rang $(70.6\%)$ . The mode of onset was acute  $(<\!48\,\mathrm{h})$  in 12 patients (70.6%) and progressive (subacute or chronic) for the other 5 patients. In one case, CVST occurred as an initial manifestation. All patients came from North China.



3.2. Neurological Features of CVST in Lupus. Headache was the most common neurological manifestation (15/17,  $88.2\%)$ ), and among these cases, 8 patients suffered from nausea/vomiting  $(47.1\%)$ . Seven  $(41.2\%)$  cases had conscious disturbance, 6  $(35.3\%)$  had edema of eyelids or conjunctiva, and 6  $(35.3\%)$  had blurred vision or diplopia. Hearing loss occurred in 2 cases  $(11.8\%)$  while 6 cases had seizures  $(35.3\%)$ . As complications, subarachnoid hemorrhage (SAH,  $4/17,23.5\%)$ , sinusitis  $(3/17,17.6\%)$ , and cerebral ischemia and infarction  $(2/17,11.8\%)$ ) were also found in SLE patients with CVST (Table 1).

All patients underwent lumbar puncture, which showed that the intracranial pressure (ICP) was higher than  $330\,\mathrm{mm}$    $180\,\mathrm{mm}\ \mathrm{H}_{2}\mathrm{O}.$  $320\,\mathrm{mm}$  $\mathrm{H}_{2}\mathrm{O},$  $(41.2\%)$  $\mathrm{H}_{2}\mathrm{O}$  180 to 320 m $\mathrm{H}_{2}\mathrm{O}.$  than 180 mm H2O.  $(52.9\%)$ mical analysis of cerebrospinal fluid (CSF) indicated that the protein level was higher than normal in 9 patients (52.9%) and glucose and chloride levels were normal. Myelin basic protein (MBP) was elevated in 5 patients. From examination of MRV images, we found that thrombosis in the transverse sinus (TS) was the most common site  $(14/17,\,82.4\%)$ , followed by sigmoid sinus (SS,  $9/17,\,52.9\%)$ ). Thrombosis in the superior sagittal sinus (SSS,  $6/17,\,35.3\%)$  and inferior sagittal sinus (ISS,  $1/17,\,5.9\%)$ ) was also seen. Twelve patients suffered from more than one sinus thrombosis  $(12/17,70.6\%)$  (Figure 1(a)).

3.3. Systemic Manifestations of SLE. Most SLE patients with CVST  $(16/17,\,94.1\%)$  also had involvement of other organs, with lupus nephritis  $(12/17,~70.6\%)$ , hemocytopenia (10/17,  $58.8\%)$ , arthritis  $(6/17,35.3\%)$ , fever  $(6/17,35.3\%)$ , rash (5/17,  $29.4\%)$ , and serositis  $(5/17,\,29.4\%)$  being the most common conditions. The mean SLE disease activity index (SLEDAI)

score (8 points of CVST excluded) of th $(12/17,70.6\%)$  $12.1\pm$  $(30.4\pm7.5)\,\mathrm{g/L}$  3.7. Twelve cases had hypoalbuminemia (12/17, 70.6%) and the mean serum albumin level of all 17 cases was (30.4 ± 7.5) g/L. The prothrombin time and activated partial thrombo- plastin time were normal in most cases. Three patients had positive antiribosomal RNA-protein (rRNP) antibody. APLs, including anticardiolipin  $(5/17,29.4\%)$  antibodies, anti-  $(6/17,\,35.3\%)$ odies (4/17,  $23.5\%,$ ), and l $(7/17,\,41.2\%)$ gulants $\beta_{2}\mathrm{GP}_{1}$  (6/17, 35.3%), were detected in 7 cases (7/17, 41.2%) in this cohort. Thrombophilia factors (protein S and protein C) were detected in 12 patients, and no protein C or S abnormality was found. Only one patient accompanied other thromboses in the veins (Table 2).

3.4. Comparison with SLE Cases without CVST. Compared with SLE patients without CVST, the incidence of thrombo-  $P~=~0.007)$  $P~=~0.043)$  and prevalence  $23.5\%$  = 0.043) were signif $(20.1\pm$  higher in SLE patients with CVST. SLE patients with CVST also $P=0.001\rangle$  $41.2\%$  $15.7\%,$ activity (SLEDAI scores)  $(12.8\pm$  3.7) when compared with SLE patients without CVST (12.8 ±  𝑃 = 0.001) (Table 3).

3.5. Treatments and Prognosis. All patients received gluco- corticoid (GC) combined with immunosuppressant treat- ment. Thirteen patients were treated by pulse therapy with

## 4. Discussion

NPLE is one of the most important manifestations of SLE, which occurs in  $14\%{\sim}80\%$ 5 of $22\%{\sim}95\%$  pediatric [6, 7] SLE patients. In 1999, ACR named NPLE into 19 categories [8]. CVST is one of the rare and severe cerebrovascular diseases, and its clinical features lack speci- ficity. Thus, rheumatologists should be aware of this rare but potentially fatal complication of SLE.

As an uncommon disorder that accounts for about  $0.5\%$  of all strokes [9], CVST presents as a variety of neurologic symptoms and can lead to severe morbidity and mortality [10]. The causes of CVST include infection, dehydration, local trauma, rheumatologic diseases such as SLE and Behc¸et’s disease (BD) [11] as well as thrombophilia including nephritic syndrome, antithrombin III deficiency, pregnancy, cancer, and use of oral contraceptives [2]. The current study showed that the incidence of CVST was about  $0.36\%$  in SLE, which is much lower than that of CVST in BD  $(7.8\%$  [11]), making this condition quite rare. It is not clear when and why CVST could happen and whether this condition is due to high SLE activity.

Several mechanisms might contribute to the formation of CVST in SLE patients. Endothelial cell injury caused by immune complex-induced vasculitis was thought to play an important role in NPLE, which includes CVST. The existence of APLs may be another cause of CVST in SLE [12]. Thrombophilia is related to some SLE complications, such as nephritic syndrome [13] and hyperfibrinogenemia caused by chronic inflammatory status and could represent another etiologic factor. Infections of the middle ear, facial skin, or intracalvarium, which were known as the most common causes of CVST [14], were secondary consequences of GC and immunosuppressant treatment and are sometimes minor factors associated with CVST in SLE. Based on our data, no related craniofacial infections were found in the SLE patients complicated with CVST. Renal involvement was documented

Organ Involvement. F: fever, R: rash, H: hemocytopenia, C: cardiac involvement, K: kidney disease, A: arthritis, P: pulmonary involvement, S: serositis, GI: gastrointestinal involvement, and CAPS: catastrophic antiphospholipid syndrome.

Site of CVST Occlusion. B: bilateral, L: left, R: right, SSS: superior sagittal sinus, ISS: inferior sagittal sinus, SS: sigmoid sinus, and TS: transverse sinus. Treatment. GC: glucocorticoid, CTX: cyclophosphamide, and MMF: mycophenolate mofetil.

in 12 cases, but few of them fulfilled the criteria of nephritic syndrome. APLs were found much more frequently  $(41.2\%)$  in this group compared to SLE patients without CVST  $(15.7\%)$ , and it hints that these molecules are involved in CVST etiology in SLE. Moreover, most cases had high disease activities and good responses to GC (systemic administration or intrathecal injection) and immunosuppressants, which supports that cerebral vasculitis might be another possible CVST mechanism.

Headaches, which can be persistent and severe and the only neurological sign, are the most frequent symptom of CVST and occur in about 90 percent of CVST patients [15]. In the current study,  $88.2\%$  patients suffered from headache, most of which increased gradually over days or weeks, while some had acute-onset mimicking or were accompanied by or even initially presenting as a SAH [16, 17]. Four cases  $(23.5\%)$  coexisted with SAH in this cohort. The cause of headaches is likely intracranial hypertension (ICH), which can also cause vomiting, papilledema, and diplopia due to involvement of the cranial nerves. Thirteen patients  $(76.5\%)$  were confirmed as ICH and the pressures in 7 cases were higher than  $330\,\mathrm{{mm}\,\,H}_{2}\mathrm{{O}}$  in the current study. Headache is also a common manifestation of NPLE [18]. Most lupus headaches are chronic and are not accompanied by ICH or other neurological signs, while headaches in the majority of CVST cases are acute or subacute. Besides headaches, about half of patients develop other neurological signs to indicate their location to practitioners [19]. SSS involvement frequently leads to paralysis and seizure, while thrombosis of the TS can be an underlying cause of acute aphasia. Isolated or multiple cranial nerve palsies are usually caused by blockade of TS or SS. Cavernous sinus syndrome, manifesting as oculomotor nerve palsies, facial pain, sensory loss in the distribution of the trigeminal nerve, or proptosis and chemosis, is often caused by infections and is very rare in SLE. Analysis of the cerebrospinal fluid through lumbar punc- ture is almost always nonspecific but is necessary to differ- entiate infections. In this study, slightly elevated CSF protein was found in more than half of patients  $(9/17,\,52.9\%)$ ). MRV is now the most sensitive tool for detecting CVST [19], but its specificity is based on meticulous knowledge of the anatomy and common variations of the cerebral sinus such as left transverse sinus hypoplasia or atresia could be seen in some patients [20]. Therefore, the diagnosis of CVST should be determined not only by imaging but also by clinical manifestations and CSF measures. SSS was reported to be the most frequently involved site  $(62.5\%)$  and TS was the second 15 $(41.2\%{\sim}44.7\%)$ ]. Infections always cause thrombosis of the cavernous sinus, TS, or SS. When thrombophilia exists, the SSS is frequently affected. Previously, there were no data concerning CVST associated with SLE. In the current study, TS was the most common site of thrombosis occurrence  $(14/17,82.4\%)$  followed by SS  $(9/17,52.9\%)$ , SSS  $(6/17,35.3\%)$ , and the inferior sagittal sinus (ISS,  $1/17,5.9\%)$ ). Twelve CVST patients had been affected by more than one sinus  $(70.6\%)$ .



The current study provided information about the sys- temic involvement of SLE cases with CVST. The data rep- resented that most cases were accompanied by renal and hematological involvement or arthritis, fever, and rash. The higher prevalence of thrombocytopenia in SLE patients with CVST might partly result from damage caused by autoanti- bodies and depletion of platelets when thrombosis occurs. The meaning of APLs was apparent and confirmed that these antibodies were involved in CVST in SLE. Concerning SLE activity, the SLEDAI of the CVST group was higher than the controls, indicating that the occurrence of CVST might parallel the systemic active situation of SLE.



The treatment of patients with CVST should include four aspects: removal of precipitating factors, administration of antithrombotic therapy, lowering of ICH, and relieving of neurological symptoms [15, 21, 22]. For SLE patients with CVST, the underlying cause of CVST is SLE. The current study showed that these cases had high SLE activity and rapid disease progression so that aggressive treatments to control SLE are necessary and important. In order to achieve clinical remission as soon as possible, most cases needed GC pulse therapy combined with CTX, which is one of the most powerful immunosuppressants. Anticoagulation was considered as the cornerstone of CVST treatment [9, 21]. Heparin [23] or LMWH was utilized by 11 cases and followed by warfarin in the current study, while SAH [24] and severe thrombocytopenia were contraindications to anticoagulation in other patients. Patients had also been treated with dehy- dration drugs and intrathecal injections of dexamethasone  $(10\,\mathrm{mg})$  to normalize their ICP.



In the past, the mortality rate of CVST in SLE reached 25 $30\%{\sim}50\%$ ]. In part due to modern neuroimaging and LMWH administration, the mortality rate was reduced to 26 $8\%{\sim}14\%$ ,]. In the current study, the mortality was  $11.8\%$  (2/17, 1 case died as the result of suicide rather than CVST). Complete or partial recovery was observed in most patients who survived (13/17,  $76.5\%)$ , while no patient experienced a relapse of CVST or had a poor outcome with permanent neurological deficits during the mean 3-year follow-up.

## 5. Conclusions

As one of the rare and severe complications of SLE, CVST usually occurs in active SLE patients and is accompanied by systemic involvement (especially thrombocytopenia). CVST in SLE may be caused by a variety of pathogenic factors, such as vasculitis and APLs. The clinical picture of CVST is nonspecific and may vary significantly due to different venous sinus involvements, which can make the diagnosis quite difficult. Lumbar puncture and MRV are necessary for establishing a diagnosis of CVST. Treatment should include controlling SLE activity, anticoagulation administration, and ICH management. Early diagnosis and prompt management of CVST and the underlying disease could significantly improve the prognosis of lupus with CVST.



## Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

## Authors’ Contribution

Li Wang and Hua Chen contributed equally to this study. Dr. Li Wang, Dr. Hua Chen, and Dr. Wenjie Zheng designed the study. Dr. Li Wang and Dr. Hua Chen analyzed the data and wrote the paper. Dr. Wenjie Zheng revised the paper. Professor Wanli Liu, Dr. Xuan Zhang, and Dr. Fengchun Zhang critically reviewed the paper and provided valuable input. Dr. Yao Zhang assessed and confirmed the neurologic data and images. All authors read and approved the final paper.

## Acknowledgments

The authors thank Lijuan Gou, Hang Yu, Mengchun Gong, Jiaxin Zhou, and all the residents and attendings, who were in charge of the cases we studied, for their contributions to save the patients’ lives and improve their life quality. This study was supported by the Chinese National High Technology Research and Development Program, Ministry of Science and Technology (2014AA020527), the Chinese Medical Asso- ciation (12040670367), and the National Natural Science Foundation of China (81302590).

## References

[1] “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,” Arthri- tis & Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[2] J. Kimber, “Cerebral venous sinus thrombosis,” QJM, vol. 95, no. 3, pp. 137–142, 2002.

[3] M. K. Lee, J. H. Kim, H. R. Kang et al., “Systemic Lupus Erythe- matosus complicated with cerebral venous sinus thrombosis: a report of two cases,” Journal of Korean Medical Science, vol. 16, no. 3, pp. 351–354, 2001.

[4] M. C. Hochberg, “Updating the American College of Rheuma- tology revised criteria for the classification of systemic lupus erythematosus,” Arthritis and Rheumatism, vol. 40, no. 9, p. 1725, 1997.

[5] E. Muscal and R. L. Brey, “Neurologic manifestations of sys- temic lupus erythematosus in children and adults,” Neurologic Clinics, vol. 28, no. 1, pp. 61–73, 2010.

[6] S. M. Benseler and E. D. Silverman, “Neuropsychiatric involve- ment in pediatric systemic lupus erythematosus,” Lupus, vol. 16, no. 8, pp. 564–571, 2007.

[7] L. T. Hiraki, S. M. Benseler, P. N. Tyrrell, D. Hebert, E. Harvey, and E. D. Silverman, “Clinical and a laboratory characteristics and long-term outcome of pediatric systemic lupus erythemato- sus: a longitudinal study,” The Journal of Pediatrics, vol. 152, no. 4, pp. 550–556, 2008.

[8] ACR Ad Hoc Committee on Neuropsychiatric Lupus Nomen- clature, “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,,” Arthritis and Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[9] A. Filippidis, E. Kapsalaki, G. Patramani, and K. N. Fountas, “Cerebral venous sinus thrombosis: review of the demograph- ics, pathophysiology, current diagnosis, and treatment,” Neuro- surgical Focus, vol. 27, no. 5, p. E3, 2009.

[10] A. B. Haghighi, R. C. Edgell, S. Cruz-Flores et al., “Mortality of cerebral venous-sinus thrombosis in a large national sample,” Stroke, vol. 43, no. 1, pp. 262–264, 2012.

[11] D. Saadoun, B. Wechsler, M. Resche-Rigon et al., “Cerebral venous thrombosis in Behcet’s disease,” Arthritis and Rheuma- tism, vol. 61, no. 4, pp. 518–526, 2009.

[12] J. Mikdashi and B. Handwerger, “Predictors of neuropsychiatric damage in systemic lupus erythematosus: data from the Mary- land lupus cohort,” Rheumatology, vol. 43, no. 12, pp. 1555–1560, 2004.

[13] H. Xu, K. Chen, D. Lin, L. Dai, H. Chen, and Z. Xu, “Cerebral venous sinus thrombosis in adult nephrotic syndrome,” Clinical Nephrology, vol. 74, no. 2, pp. 144–149, 2010.

[14] J. Stam, “Thrombosis of the cerebral veins and sinuses,” The New England Journal of Medicine, vol. 352, no. 17, pp. 1791–1798, 2005. [15] J. M. Ferro, P. Canh˜ao, J. Stam, M.-G. Bousser, and F. Bari- nagarrementeria, “Prognosis of cerebral vein and dural sinus thrombosis: results of the International Study on Cerebral Vein and dural sinus Thrombosis (ISCVT),” Stroke, vol. 35, no. 3, pp. 664–670, 2004.

[16] Y. Kato, H. Takeda, D. Furuya et al., “Subarachnoid hemorrhage as the initial presentation of cerebral venous thrombosis,” Internal Medicine, vol. 49, no. 5, pp. 467–470, 2010.

[17] S. Panda, D. K. Prashantha, S. Ravi Shankar, and D. Nagaraja, “Localized convexity subarachnoid haemorrhage—a sign of early cerebral venous sinus thrombosis,” European Journal of Neurology, vol. 17, no. 10, pp. 1249–1258, 2010.

[18] ACR Ad Hoc Committee on Neuropsychiatric Lupus Nomen- clature, “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,” Arthritis and Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[19] M.-G. Bousser and J. M. Ferro, “Cerebral venous thrombosis: an update,” The Lancet Neurology, vol. 6, no. 2, pp. 162–170, 2007.

[20] A. Zouaoui and G. Hidden, “Cerebral venous sinuses: anatomi- cal variants or thrombosis?” Acta Anatomica, vol. 133, no. 4, pp. 318–324, 1988.

[21] K. Einh¨aupl, M.-G. Bousser, S. F. de Bruijn et al., “EFNS guide- line on the treatment of cerebral venous and sinus thrombosis,” European Journal of Neurology, vol. 13, no. 6, pp. 553–559, 2006. [22] F. Masuhr and K. Einh¨aupl, “Treatment of cerebral venous and sinus thrombosis,” Frontiers of Neurology and Neuroscience, vol. 23, pp. 132–143, 2008.

[23] S. Mehraein, K. Schmidtke, A. Villringer, J. M. Valdueza, and F. Masuhr, “Heparin treatment in cerebral sinus and venous thrombosis: patients at risk of fatal outcome,” Cerebrovascular Diseases, vol. 15, no. 1-2, pp. 17–21, 2003.

[24] B. Fuentes, P. Mart´ınez-S´anchez, P. G. Raya, M. J. Abenza, and E. D. Tejedor, “Cerebral venous sinus thrombosis associated with cerebral hemorrhage: is anticoagulant treatment safe?” Neurologist, vol. 17, no. 4, pp. 208–210, 2011.

[25] R. D. Nagpal, “Dural sinus and cerebral venous thrombosis,” Neurosurgical Review, vol. 6, no. 3, pp. 155–160, 1983.

[26] I. Gosk-Bierska, W. Wysokinski, R. D. Brown Jr. et al., “Cerebral venous sinus thrombosis: incidence of venous thrombosis recurrence and survival,” Neurology, vol. 67, no. 5, pp. 814–819, 2006.

[27] F. Dentali, M. Crowther, and W. Ageno, “Thrombophilic abnor- malities, oral contraceptives, and risk of cerebral vein thrombo- sis: a meta-analysis,” Blood, vol. 107, no. 7, pp. 2766–2773, 2006.

Hindawi Publishing Corporationhttp://www.hindawi.comVolume 2014 Disease Markers
# Clinical Study Clinical Characteristics of Cerebral Venous Sinus Thrombosis in Patients with Systemic Lupus Erythematosus: A Single-Centre Experience in China

Li Wang, Hua Chen, Yao Zhang, Wanli Liu, Wenjie Zheng, Xuan Zhang, and Fengchun Zhang



Key Laboratory of Rheumatology and Clinical Immunology, Department of Rheumatology and Clinical Immunology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, Ministry of Education, No. 1 Shuaifuyuan, Beijing 100730, China Department of Neurology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences, No. 1 Shuaifuyuan, Beijing 100730, China MOE Key Laboratory of Protein Science, School of Life Sciences, Tsinghua University, Beijing 100084, China Correspondence should be addressed to Wenjie Zheng; wenjzheng@gmail.com Received 22 August 2014; Accepted 17 November 2014 Academic Editor: Guixiu Shi Copyright  $^\copyright$  2015 Li Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

   

  

 



Clinical characteristics of systemic lupus erythematosus (SLE) patients complicated with cerebral venous sinus thrombosis (CVST) between 2000 and 2013 were analyzed through this retrospective, single-centre study. Of 4747 hospitalized SLE patients, 17  $(0.36\%,$ , 12 females, average age 30) had CVST. Headache  $(88.2\%)$  was the most common neurological symptom followed by nausea or vomiting  $(47.1\%)$ , conscious disturbance  $(41.2\%)$ , edema of eyelids or conjunctiva  $(35.3\%)$ , blurred vision or diplopia  $(35.3\%)$ , and seizure  $(35.3\%)$ . Increased intracranial pressure (ICP) occurred in 13 cases  $(76.5\%)$ . Magnetic resonance venography (MRV) detected thrombosis in the transverse  $(82.4\%)$ , sigmoid  $(52.9\%)$ , and sagittal  $(35.3\%)$  sinuses, with frequent  $(70.6\%)$  multiple sinus occlusions. Compared to SLE patients without CVST, SLE patients with CVST had a higher prevalence of thrombocytopenia and positive antiphospholipid antibodies and a higher SLE disease activity index (SLEDAI) score. 13 patients achieved improvement following glucocorticoids and immunosuppressants treatment, as well as anticoagulants, while 3 patients died at the hospital. CVST is relatively rare in SLE and tends to occur in active lupus patients. Intracranial hypertension is common but nonspecific clinical feature, so MRV evaluation is necessary to establish a diagnosis. Aggressive treatment for the rapid control of SLE activity combined with anticoagulants can improve the prognosis.

## 1. Introduction

Systemic lupus erythematosus (SLE) is a heterogeneous, mul- tisystem autoimmune disease associated with the production of antibodies against self-antigens. Its pathologic basis is vasculitis induced by immune complex, which suggests that blood vessels might be an important target of autoimmunity overactivity. As a result, there is a wide variation in ves- sel involvement in SLE, including arteriostenosis and deep venous thrombosis, while glomerulonephritis and defused alveolar hemorrhage can also occur to a certain extent. Cerebral venous sinus thrombosis (CVST) is an uncommon disorder in SLE that is a manifestation of overlapping neu- ropsychiatric lupus (NPLE) [1] and vessel disease. CVST has been rarely reported and is potentially fatal to the patient due to the acute neurological deterioration that can occur. As such, practitioners should be familiar with this condition, so the correct diagnosis can be made as soon as possible to improve patient prognosis.



The etiology of CVST is complicated [2]. In SLE, the main culprits are vascular injuries caused by vasculitis. Antiphos- pholipid antibodies (APLs) and prothrombotic tendencies might also be responsible for CVST [3].

By investigating the clinical features of CVST in SLE and analyzing the differences between SLE patients with and without CVST, this study seeks to explore the characteristics and risk factors for CVST in SLE patients.

## 2. Patients and Methods

2.1. Patients. 4747 SLE patients admitted to the Peking Union Medical College Hospital from January 2000 to December 2013 were reviewed. All patients were diagnosed as having SLE according to 1997 American College of Rheumatology (ACR) revised classification criteria [4]. In the SLE cohort, after exclusion of patients with infectious and traumatic CVST or incomplete data, we identified 17 cases  $(0.36\%)$  complicated with CVST as indicated by clinical features, lumbar puncture, imaging materials (magnetic resonance venography (MRV)), and at least one neurologist’s confir- mation. For each $(n~=~51)$  $\begin{array}{r l r}{(n}&{{}=}&{17)}\end{array}$ =17), three age- and sex- matched controls (𝑛 = 51) were randomly selected from the contemporaneously admitted SLE patients without CVST. The local institutional review board approved the study. Because the study was based on a review of medical records that had been obtained for clinical purposes, the requirement for written informed consent was waived.

2.2. Statistical Analyses. SPSS version 16.0 (SPSS Inc., Chic- ago, USA) was used to statistically analyze the data. Numer- ical data and categorical data were expressed as mean  $\pm\ \mathrm{SD}$  (range) and percentage, respectively. The significance was estimated by Student’s 𝑡-test, Pearson’s chi-sq $^{<5}$ re, or Fishe $<$  exact test (when expected frequencies were <5). 𝑃 values < 0.05 were considered to be statistically significant.

## 3. Results

3.1. Demographic Characteristics. Among 17 SLE patients with CVST, 12  $(70.6\%)$  were women. The median age was 30 years old (range 12–52) and the SLE disease duration was 12 months (rang $(70.6\%)$ . The mode of onset was acute  $(<\!48\,\mathrm{h})$  in 12 patients (70.6%) and progressive (subacute or chronic) for the other 5 patients. In one case, CVST occurred as an initial manifestation. All patients came from North China.



3.2. Neurological Features of CVST in Lupus. Headache was the most common neurological manifestation (15/17,  $88.2\%)$ ), and among these cases, 8 patients suffered from nausea/vomiting  $(47.1\%)$ . Seven  $(41.2\%)$  cases had conscious disturbance, 6  $(35.3\%)$  had edema of eyelids or conjunctiva, and 6  $(35.3\%)$  had blurred vision or diplopia. Hearing loss occurred in 2 cases  $(11.8\%)$  while 6 cases had seizures  $(35.3\%)$ . As complications, subarachnoid hemorrhage (SAH,  $4/17,23.5\%)$ , sinusitis  $(3/17,17.6\%)$ , and cerebral ischemia and infarction  $(2/17,11.8\%)$ ) were also found in SLE patients with CVST (Table 1).

All patients underwent lumbar puncture, which showed that the intracranial pressure (ICP) was higher than  $330\,\mathrm{mm}$    $180\,\mathrm{mm}\ \mathrm{H}_{2}\mathrm{O}.$  $320\,\mathrm{mm}$  $\mathrm{H}_{2}\mathrm{O},$  $(41.2\%)$  $\mathrm{H}_{2}\mathrm{O}$  180 to 320 m $\mathrm{H}_{2}\mathrm{O}.$  than 180 mm H2O.  $(52.9\%)$ mical analysis of cerebrospinal fluid (CSF) indicated that the protein level was higher than normal in 9 patients (52.9%) and glucose and chloride levels were normal. Myelin basic protein (MBP) was elevated in 5 patients. From examination of MRV images, we found that thrombosis in the transverse sinus (TS) was the most common site  $(14/17,\,82.4\%)$ , followed by sigmoid sinus (SS,  $9/17,\,52.9\%)$ ). Thrombosis in the superior sagittal sinus (SSS,  $6/17,\,35.3\%)$  and inferior sagittal sinus (ISS,  $1/17,\,5.9\%)$ ) was also seen. Twelve patients suffered from more than one sinus thrombosis  $(12/17,70.6\%)$  (Figure 1(a)).

3.3. Systemic Manifestations of SLE. Most SLE patients with CVST  $(16/17,\,94.1\%)$  also had involvement of other organs, with lupus nephritis  $(12/17,~70.6\%)$ , hemocytopenia (10/17,  $58.8\%)$ , arthritis  $(6/17,35.3\%)$ , fever  $(6/17,35.3\%)$ , rash (5/17,  $29.4\%)$ , and serositis  $(5/17,\,29.4\%)$  being the most common conditions. The mean SLE disease activity index (SLEDAI)

score (8 points of CVST excluded) of th $(12/17,70.6\%)$  $12.1\pm$  $(30.4\pm7.5)\,\mathrm{g/L}$  3.7. Twelve cases had hypoalbuminemia (12/17, 70.6%) and the mean serum albumin level of all 17 cases was (30.4 ± 7.5) g/L. The prothrombin time and activated partial thrombo- plastin time were normal in most cases. Three patients had positive antiribosomal RNA-protein (rRNP) antibody. APLs, including anticardiolipin  $(5/17,29.4\%)$  antibodies, anti-  $(6/17,\,35.3\%)$ odies (4/17,  $23.5\%,$ ), and l $(7/17,\,41.2\%)$ gulants $\beta_{2}\mathrm{GP}_{1}$  (6/17, 35.3%), were detected in 7 cases (7/17, 41.2%) in this cohort. Thrombophilia factors (protein S and protein C) were detected in 12 patients, and no protein C or S abnormality was found. Only one patient accompanied other thromboses in the veins (Table 2).

3.4. Comparison with SLE Cases without CVST. Compared with SLE patients without CVST, the incidence of thrombo-  $P~=~0.007)$  $P~=~0.043)$  and prevalence  $23.5\%$  = 0.043) were signif $(20.1\pm$  higher in SLE patients with CVST. SLE patients with CVST also $P=0.001\rangle$  $41.2\%$  $15.7\%,$ activity (SLEDAI scores)  $(12.8\pm$  3.7) when compared with SLE patients without CVST (12.8 ±  𝑃 = 0.001) (Table 3).

3.5. Treatments and Prognosis. All patients received gluco- corticoid (GC) combined with immunosuppressant treat- ment. Thirteen patients were treated by pulse therapy with

## 4. Discussion

NPLE is one of the most important manifestations of SLE, which occurs in  $14\%{\sim}80\%$ 5 of $22\%{\sim}95\%$  pediatric [6, 7] SLE patients. In 1999, ACR named NPLE into 19 categories [8]. CVST is one of the rare and severe cerebrovascular diseases, and its clinical features lack speci- ficity. Thus, rheumatologists should be aware of this rare but potentially fatal complication of SLE.

As an uncommon disorder that accounts for about  $0.5\%$  of all strokes [9], CVST presents as a variety of neurologic symptoms and can lead to severe morbidity and mortality [10]. The causes of CVST include infection, dehydration, local trauma, rheumatologic diseases such as SLE and Behc¸et’s disease (BD) [11] as well as thrombophilia including nephritic syndrome, antithrombin III deficiency, pregnancy, cancer, and use of oral contraceptives [2]. The current study showed that the incidence of CVST was about  $0.36\%$  in SLE, which is much lower than that of CVST in BD  $(7.8\%$  [11]), making this condition quite rare. It is not clear when and why CVST could happen and whether this condition is due to high SLE activity.

Several mechanisms might contribute to the formation of CVST in SLE patients. Endothelial cell injury caused by immune complex-induced vasculitis was thought to play an important role in NPLE, which includes CVST. The existence of APLs may be another cause of CVST in SLE [12]. Thrombophilia is related to some SLE complications, such as nephritic syndrome [13] and hyperfibrinogenemia caused by chronic inflammatory status and could represent another etiologic factor. Infections of the middle ear, facial skin, or intracalvarium, which were known as the most common causes of CVST [14], were secondary consequences of GC and immunosuppressant treatment and are sometimes minor factors associated with CVST in SLE. Based on our data, no related craniofacial infections were found in the SLE patients complicated with CVST. Renal involvement was documented

Organ Involvement. F: fever, R: rash, H: hemocytopenia, C: cardiac involvement, K: kidney disease, A: arthritis, P: pulmonary involvement, S: serositis, GI: gastrointestinal involvement, and CAPS: catastrophic antiphospholipid syndrome.

Site of CVST Occlusion. B: bilateral, L: left, R: right, SSS: superior sagittal sinus, ISS: inferior sagittal sinus, SS: sigmoid sinus, and TS: transverse sinus. Treatment. GC: glucocorticoid, CTX: cyclophosphamide, and MMF: mycophenolate mofetil.

in 12 cases, but few of them fulfilled the criteria of nephritic syndrome. APLs were found much more frequently  $(41.2\%)$  in this group compared to SLE patients without CVST  $(15.7\%)$ , and it hints that these molecules are involved in CVST etiology in SLE. Moreover, most cases had high disease activities and good responses to GC (systemic administration or intrathecal injection) and immunosuppressants, which supports that cerebral vasculitis might be another possible CVST mechanism.

Headaches, which can be persistent and severe and the only neurological sign, are the most frequent symptom of CVST and occur in about 90 percent of CVST patients [15]. In the current study,  $88.2\%$  patients suffered from headache, most of which increased gradually over days or weeks, while some had acute-onset mimicking or were accompanied by or even initially presenting as a SAH [16, 17]. Four cases  $(23.5\%)$  coexisted with SAH in this cohort. The cause of headaches is likely intracranial hypertension (ICH), which can also cause vomiting, papilledema, and diplopia due to involvement of the cranial nerves. Thirteen patients  $(76.5\%)$  were confirmed as ICH and the pressures in 7 cases were higher than  $330\,\mathrm{{mm}\,\,H}_{2}\mathrm{{O}}$  in the current study. Headache is also a common manifestation of NPLE [18]. Most lupus headaches are chronic and are not accompanied by ICH or other neurological signs, while headaches in the majority of CVST cases are acute or subacute. Besides headaches, about half of patients develop other neurological signs to indicate their location to practitioners [19]. SSS involvement frequently leads to paralysis and seizure, while thrombosis of the TS can be an underlying cause of acute aphasia. Isolated or multiple cranial nerve palsies are usually caused by blockade of TS or SS. Cavernous sinus syndrome, manifesting as oculomotor nerve palsies, facial pain, sensory loss in the distribution of the trigeminal nerve, or proptosis and chemosis, is often caused by infections and is very rare in SLE. Analysis of the cerebrospinal fluid through lumbar punc- ture is almost always nonspecific but is necessary to differ- entiate infections. In this study, slightly elevated CSF protein was found in more than half of patients  $(9/17,\,52.9\%)$ ). MRV is now the most sensitive tool for detecting CVST [19], but its specificity is based on meticulous knowledge of the anatomy and common variations of the cerebral sinus such as left transverse sinus hypoplasia or atresia could be seen in some patients [20]. Therefore, the diagnosis of CVST should be determined not only by imaging but also by clinical manifestations and CSF measures. SSS was reported to be the most frequently involved site  $(62.5\%)$  and TS was the second 15 $(41.2\%{\sim}44.7\%)$ ]. Infections always cause thrombosis of the cavernous sinus, TS, or SS. When thrombophilia exists, the SSS is frequently affected. Previously, there were no data concerning CVST associated with SLE. In the current study, TS was the most common site of thrombosis occurrence  $(14/17,82.4\%)$  followed by SS  $(9/17,52.9\%)$ , SSS  $(6/17,35.3\%)$ , and the inferior sagittal sinus (ISS,  $1/17,5.9\%)$ ). Twelve CVST patients had been affected by more than one sinus  $(70.6\%)$ .



The current study provided information about the sys- temic involvement of SLE cases with CVST. The data rep- resented that most cases were accompanied by renal and hematological involvement or arthritis, fever, and rash. The higher prevalence of thrombocytopenia in SLE patients with CVST might partly result from damage caused by autoanti- bodies and depletion of platelets when thrombosis occurs. The meaning of APLs was apparent and confirmed that these antibodies were involved in CVST in SLE. Concerning SLE activity, the SLEDAI of the CVST group was higher than the controls, indicating that the occurrence of CVST might parallel the systemic active situation of SLE.



The treatment of patients with CVST should include four aspects: removal of precipitating factors, administration of antithrombotic therapy, lowering of ICH, and relieving of neurological symptoms [15, 21, 22]. For SLE patients with CVST, the underlying cause of CVST is SLE. The current study showed that these cases had high SLE activity and rapid disease progression so that aggressive treatments to control SLE are necessary and important. In order to achieve clinical remission as soon as possible, most cases needed GC pulse therapy combined with CTX, which is one of the most powerful immunosuppressants. Anticoagulation was considered as the cornerstone of CVST treatment [9, 21]. Heparin [23] or LMWH was utilized by 11 cases and followed by warfarin in the current study, while SAH [24] and severe thrombocytopenia were contraindications to anticoagulation in other patients. Patients had also been treated with dehy- dration drugs and intrathecal injections of dexamethasone  $(10\,\mathrm{mg})$  to normalize their ICP.



In the past, the mortality rate of CVST in SLE reached 25 $30\%{\sim}50\%$ ]. In part due to modern neuroimaging and LMWH administration, the mortality rate was reduced to 26 $8\%{\sim}14\%$ ,]. In the current study, the mortality was  $11.8\%$  (2/17, 1 case died as the result of suicide rather than CVST). Complete or partial recovery was observed in most patients who survived (13/17,  $76.5\%)$ , while no patient experienced a relapse of CVST or had a poor outcome with permanent neurological deficits during the mean 3-year follow-up.

## 5. Conclusions

As one of the rare and severe complications of SLE, CVST usually occurs in active SLE patients and is accompanied by systemic involvement (especially thrombocytopenia). CVST in SLE may be caused by a variety of pathogenic factors, such as vasculitis and APLs. The clinical picture of CVST is nonspecific and may vary significantly due to different venous sinus involvements, which can make the diagnosis quite difficult. Lumbar puncture and MRV are necessary for establishing a diagnosis of CVST. Treatment should include controlling SLE activity, anticoagulation administration, and ICH management. Early diagnosis and prompt management of CVST and the underlying disease could significantly improve the prognosis of lupus with CVST.



## Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

## Authors’ Contribution

Li Wang and Hua Chen contributed equally to this study. Dr. Li Wang, Dr. Hua Chen, and Dr. Wenjie Zheng designed the study. Dr. Li Wang and Dr. Hua Chen analyzed the data and wrote the paper. Dr. Wenjie Zheng revised the paper. Professor Wanli Liu, Dr. Xuan Zhang, and Dr. Fengchun Zhang critically reviewed the paper and provided valuable input. Dr. Yao Zhang assessed and confirmed the neurologic data and images. All authors read and approved the final paper.

## Acknowledgments

The authors thank Lijuan Gou, Hang Yu, Mengchun Gong, Jiaxin Zhou, and all the residents and attendings, who were in charge of the cases we studied, for their contributions to save the patients’ lives and improve their life quality. This study was supported by the Chinese National High Technology Research and Development Program, Ministry of Science and Technology (2014AA020527), the Chinese Medical Asso- ciation (12040670367), and the National Natural Science Foundation of China (81302590).

## References

[1] “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,” Arthri- tis & Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[2] J. Kimber, “Cerebral venous sinus thrombosis,” QJM, vol. 95, no. 3, pp. 137–142, 2002.

[3] M. K. Lee, J. H. Kim, H. R. Kang et al., “Systemic Lupus Erythe- matosus complicated with cerebral venous sinus thrombosis: a report of two cases,” Journal of Korean Medical Science, vol. 16, no. 3, pp. 351–354, 2001.

[4] M. C. Hochberg, “Updating the American College of Rheuma- tology revised criteria for the classification of systemic lupus erythematosus,” Arthritis and Rheumatism, vol. 40, no. 9, p. 1725, 1997.

[5] E. Muscal and R. L. Brey, “Neurologic manifestations of sys- temic lupus erythematosus in children and adults,” Neurologic Clinics, vol. 28, no. 1, pp. 61–73, 2010.

[6] S. M. Benseler and E. D. Silverman, “Neuropsychiatric involve- ment in pediatric systemic lupus erythematosus,” Lupus, vol. 16, no. 8, pp. 564–571, 2007.

[7] L. T. Hiraki, S. M. Benseler, P. N. Tyrrell, D. Hebert, E. Harvey, and E. D. Silverman, “Clinical and a laboratory characteristics and long-term outcome of pediatric systemic lupus erythemato- sus: a longitudinal study,” The Journal of Pediatrics, vol. 152, no. 4, pp. 550–556, 2008.

[8] ACR Ad Hoc Committee on Neuropsychiatric Lupus Nomen- clature, “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,,” Arthritis and Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[9] A. Filippidis, E. Kapsalaki, G. Patramani, and K. N. Fountas, “Cerebral venous sinus thrombosis: review of the demograph- ics, pathophysiology, current diagnosis, and treatment,” Neuro- surgical Focus, vol. 27, no. 5, p. E3, 2009.

[10] A. B. Haghighi, R. C. Edgell, S. Cruz-Flores et al., “Mortality of cerebral venous-sinus thrombosis in a large national sample,” Stroke, vol. 43, no. 1, pp. 262–264, 2012.

[11] D. Saadoun, B. Wechsler, M. Resche-Rigon et al., “Cerebral venous thrombosis in Behcet’s disease,” Arthritis and Rheuma- tism, vol. 61, no. 4, pp. 518–526, 2009.

[12] J. Mikdashi and B. Handwerger, “Predictors of neuropsychiatric damage in systemic lupus erythematosus: data from the Mary- land lupus cohort,” Rheumatology, vol. 43, no. 12, pp. 1555–1560, 2004.

[13] H. Xu, K. Chen, D. Lin, L. Dai, H. Chen, and Z. Xu, “Cerebral venous sinus thrombosis in adult nephrotic syndrome,” Clinical Nephrology, vol. 74, no. 2, pp. 144–149, 2010.

[14] J. Stam, “Thrombosis of the cerebral veins and sinuses,” The New England Journal of Medicine, vol. 352, no. 17, pp. 1791–1798, 2005. [15] J. M. Ferro, P. Canh˜ao, J. Stam, M.-G. Bousser, and F. Bari- nagarrementeria, “Prognosis of cerebral vein and dural sinus thrombosis: results of the International Study on Cerebral Vein and dural sinus Thrombosis (ISCVT),” Stroke, vol. 35, no. 3, pp. 664–670, 2004.

[16] Y. Kato, H. Takeda, D. Furuya et al., “Subarachnoid hemorrhage as the initial presentation of cerebral venous thrombosis,” Internal Medicine, vol. 49, no. 5, pp. 467–470, 2010.

[17] S. Panda, D. K. Prashantha, S. Ravi Shankar, and D. Nagaraja, “Localized convexity subarachnoid haemorrhage—a sign of early cerebral venous sinus thrombosis,” European Journal of Neurology, vol. 17, no. 10, pp. 1249–1258, 2010.

[18] ACR Ad Hoc Committee on Neuropsychiatric Lupus Nomen- clature, “The American College of Rheumatology nomenclature and case definitions for neuropsychiatric lupus syndromes,” Arthritis and Rheumatism, vol. 42, no. 4, pp. 599–608, 1999.

[19] M.-G. Bousser and J. M. Ferro, “Cerebral venous thrombosis: an update,” The Lancet Neurology, vol. 6, no. 2, pp. 162–170, 2007.

[20] A. Zouaoui and G. Hidden, “Cerebral venous sinuses: anatomi- cal variants or thrombosis?” Acta Anatomica, vol. 133, no. 4, pp. 318–324, 1988.

[21] K. Einh¨aupl, M.-G. Bousser, S. F. de Bruijn et al., “EFNS guide- line on the treatment of cerebral venous and sinus thrombosis,” European Journal of Neurology, vol. 13, no. 6, pp. 553–559, 2006. [22] F. Masuhr and K. Einh¨aupl, “Treatment of cerebral venous and sinus thrombosis,” Frontiers of Neurology and Neuroscience, vol. 23, pp. 132–143, 2008.

[23] S. Mehraein, K. Schmidtke, A. Villringer, J. M. Valdueza, and F. Masuhr, “Heparin treatment in cerebral sinus and venous thrombosis: patients at risk of fatal outcome,” Cerebrovascular Diseases, vol. 15, no. 1-2, pp. 17–21, 2003.

[24] B. Fuentes, P. Mart´ınez-S´anchez, P. G. Raya, M. J. Abenza, and E. D. Tejedor, “Cerebral venous sinus thrombosis associated with cerebral hemorrhage: is anticoagulant treatment safe?” Neurologist, vol. 17, no. 4, pp. 208–210, 2011.

[25] R. D. Nagpal, “Dural sinus and cerebral venous thrombosis,” Neurosurgical Review, vol. 6, no. 3, pp. 155–160, 1983.

[26] I. Gosk-Bierska, W. Wysokinski, R. D. Brown Jr. et al., “Cerebral venous sinus thrombosis: incidence of venous thrombosis recurrence and survival,” Neurology, vol. 67, no. 5, pp. 814–819, 2006.

[27] F. Dentali, M. Crowther, and W. Ageno, “Thrombophilic abnor- malities, oral contraceptives, and risk of cerebral vein thrombo- sis: a meta-analysis,” Blood, vol. 107, no. 7, pp. 2766–2773, 2006.

Hindawi Publishing Corporationhttp://www.hindawi.comVolume 2014 Disease Markers
# Correction

Article title: Creating compliance value through informal work procedures Authors: Nzewi, O., Sibanda, M., Sikhosana, N., & Sentiwe, M.



## Journal: Cogent Social Sciences

Bibliometrics: Volume 4, Number 1, pages 1-17 DOI: https://doi.org/10.1080/23311886.2018.1551828



Additional information is available at the end of the article

When this article was first published online, acknowledgment section was wrongly included. It has now been removed in the published version.

© 2019 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.

You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. No additional restrictions

Cogent Social Sciences (ISSN: 2331-1886) is published by Cogent OA, part of Taylor & Francis Group.

## Publishing with Cogent OA ensures:

•Immediate, universal access to your article on publication

•High visibility and discoverability via the Cogent OA website as well as Taylor & Francis Online

•Download and citation statistics for your article

•Rapid online publication

•Input from, and dialog with, expert editors and editorial boards

•Retention of full copyright of your article

•Guaranteed legacy preservation of your article

•Discounts and waivers for authors in developing regions

## Submit your manuscript to a Cogent OA journal at www.CogentOA.com
# Correction

Article title: Creating compliance value through informal work procedures Authors: Nzewi, O., Sibanda, M., Sikhosana, N., & Sentiwe, M.



## Journal: Cogent Social Sciences

Bibliometrics: Volume 4, Number 1, pages 1-17 DOI: https://doi.org/10.1080/23311886.2018.1551828



Additional information is available at the end of the article

When this article was first published online, acknowledgment section was wrongly included. It has now been removed in the published version.

© 2019 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.

You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. No additional restrictions

Cogent Social Sciences (ISSN: 2331-1886) is published by Cogent OA, part of Taylor & Francis Group.

## Publishing with Cogent OA ensures:

•Immediate, universal access to your article on publication

•High visibility and discoverability via the Cogent OA website as well as Taylor & Francis Online

•Download and citation statistics for your article

•Rapid online publication

•Input from, and dialog with, expert editors and editorial boards

•Retention of full copyright of your article

•Guaranteed legacy preservation of your article

•Discounts and waivers for authors in developing regions

## Submit your manuscript to a Cogent OA journal at www.CogentOA.com
# Correction

Article title: Creating compliance value through informal work procedures Authors: Nzewi, O., Sibanda, M., Sikhosana, N., & Sentiwe, M.



## Journal: Cogent Social Sciences

Bibliometrics: Volume 4, Number 1, pages 1-17 DOI: https://doi.org/10.1080/23311886.2018.1551828



Additional information is available at the end of the article

When this article was first published online, acknowledgment section was wrongly included. It has now been removed in the published version.

© 2019 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.

You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. No additional restrictions

Cogent Social Sciences (ISSN: 2331-1886) is published by Cogent OA, part of Taylor & Francis Group.

## Publishing with Cogent OA ensures:

•Immediate, universal access to your article on publication

•High visibility and discoverability via the Cogent OA website as well as Taylor & Francis Online

•Download and citation statistics for your article

•Rapid online publication

•Input from, and dialog with, expert editors and editorial boards

•Retention of full copyright of your article

•Guaranteed legacy preservation of your article

•Discounts and waivers for authors in developing regions

## Submit your manuscript to a Cogent OA journal at www.CogentOA.com
# Correction

Article title: Creating compliance value through informal work procedures Authors: Nzewi, O., Sibanda, M., Sikhosana, N., & Sentiwe, M.



## Journal: Cogent Social Sciences

Bibliometrics: Volume 4, Number 1, pages 1-17 DOI: https://doi.org/10.1080/23311886.2018.1551828



Additional information is available at the end of the article

When this article was first published online, acknowledgment section was wrongly included. It has now been removed in the published version.

© 2019 The Author(s). This open access article is distributed under a Creative Commons Attribution (CC-BY) 4.0 license.

You are free to: Share — copy and redistribute the material in any medium or format. Adapt — remix, transform, and build upon the material for any purpose, even commercially.

The licensor cannot revoke these freedoms as long as you follow the license terms.

Under the following terms: Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use. No additional restrictions

Cogent Social Sciences (ISSN: 2331-1886) is published by Cogent OA, part of Taylor & Francis Group.

## Publishing with Cogent OA ensures:

•Immediate, universal access to your article on publication

•High visibility and discoverability via the Cogent OA website as well as Taylor & Francis Online

•Download and citation statistics for your article

•Rapid online publication

•Input from, and dialog with, expert editors and editorial boards

•Retention of full copyright of your article

•Guaranteed legacy preservation of your article

•Discounts and waivers for authors in developing regions

## Submit your manuscript to a Cogent OA journal at www.CogentOA.com


# Analyzing Passenger Incidence Behavior   in Heterogeneous Transit Services   Using Smartcard Data and   Schedule-Based Assignment

Michael Frumin and Jinhua Zhao

Passenger incidence (station arrival) behavior has been studied primar- ily to understand how changes to a transit service will affect passenger  waiting times. The impact of one intervention (e.g., increasing frequency)  could be overestimated when compared with another (e.g., improving  reliability), depending on the assumption of incidence behavior. Under- standing passenger incidence allows management decisions to be based on  realistic behavioral assumptions. Earlier studies on passenger incidence  chose their data samples from stations with a single service pattern such  that the linking of passengers to services was straightforward. This choice  of data samples simplifies the analysis but heavily limits the stations that  can be studied. In any moderately complex network, many stations may  have more than one service pattern. This limitation prevents the method  from being systematically applied to the whole network and constrains its  use in practice. This paper considers incidence behavior in stations with  heterogeneous services and proposes a method for estimating incidence  headway and waiting time by integrating disaggregate smartcard data  with published timetables using schedule-based assignment. This method  is applied to stations in the entire London Overground to demonstrate  its practicality; incidence behavior varies across the network and across  times of day and reflects headways and reliability. Incidence is much  less timetable-dependent on the North London Line than on the other  lines because of shorter headways and poorer reliability. Where incidence  is timetable-dependent, passengers reduce their mean scheduled waiting  time by more than 3 min compared with random incidence.

This paper is concerned with passenger incidence behavior, which  is defined as the act or event of a person’s being incident to a public  transport service with intent to use that service. A lexical convention  is established here to avoid ambiguity between passenger incidence  to public transport services and arrival at certain destinations after  using public transport services.

Passenger incidence behavior has been studied primarily for the  sake of understanding how changes to a public transport service will  affect passenger waiting times. It is also of interest because it affects  the relationship between the departure times of public transport  vehicles and the passenger loads on those vehicles. Understanding  passenger incidence behaviors allows management interventions to  be based on realistic behavioral assumptions. As pointed out in one  of the seminal investigations on the topic by Bowman and Turnquist,  the effects on passenger waiting time of one particular intervention  (e.g., increasing frequency) could be overestimated compared with a  different type of intervention (e.g., improving reliability),  depending  on which assumptions are made about incidence behavior (1).



Earlier studies on passenger incidence behavior chose their data  samples from stations or stops with a single service pattern such  that the linking of passengers to scheduled or actual services was  straightforward. This choice of data samples simplifies the analysis  and may suffice for an academic study. But such simplification also  heavily limits the stations or stops that can be studied in the transit  network. In any moderately complex network, many stations may  have more than one service pattern. This limitation prevents the  method from being systematically applied to the entire network and  therefore limits its use in transit management practice. In environ- ments in which passengers at a given station have a choice of ser- vices, a more sophisticated approach is needed to study passenger  incidence behavior.

This paper is concerned with the relationship between the times  of passenger incidence and published timetables in a rail network  with heterogeneous service patterns. It proposes a method to study  this relationship by integrating disaggregate passenger journey data  from automatic fare collection (AFC) smartcard systems with pub- lished timetables using schedule-based assignment. The purpose of  this paper is twofold: first, to develop a method that contributes to  the study of passenger incidence behavior across a railway network  with heterogeneous service patterns and frequencies using pub- lished timetables and AFC data; and second, to apply this method  to the London Overground to demonstrate its practicality and shed  light on the incidence behavior of its passengers.

### Literature

Previous research has identified a rich set of passenger incidence behav- iors and related them to certain aspects of public transport services.  Using manually and automatically collected data sources, research  has investigated the diversity of the behavior cross- sectionally and the  consistency of such behaviors longitudinally over time. It has been  found that the randomness of passenger incidence behavior is highly  dependent on the service headway and the reliability of the departure  time of the service to which passengers are incident.



After briefly introducing the random incidence model, which is  often assumed to hold at short headways, the balance of this section  reviews six studies of passenger incidence behavior that are moti- vated by understanding the relationships between service headway,  service reliability, passenger incidence behavior, and passenger  waiting time in a more nuanced fashion than is embedded in the  random incidence assumption (2). Three of these studies depend on  manually collected data, two studies use data from AFC systems,  and one study analyzes the issue purely theoretically. These studies  reveal much about passenger incidence behavior, but all are found  to be limited in their general applicability by the methods with  which they collect information about passengers and the services  those passengers intend to use.

### random Passenger incidence Behavior

One characterization of passenger incidence behavior is that of ran- dom incidence (3). The key assumption underlying the random inci- dence model is that the process of passenger arrivals to the public  transport service is independent from the vehicle departure process  of the service. This implies that passengers become incident to the  service at a random time, and thus the instantaneous rate of passen- ger arrivals to the service is uniform over a given period of time. Let  W and $H$  be random variables representing passenger waiting times  and service headways, respectively. Under the random incidence  assumption and the assumption that vehicle capacity is not a binding  constraint, a classic result of transportation science is that

## 
$$
E{\bigl(}W{\bigr)}\!=\!{\frac{E\!\!\left[H^{2}\right]}{2E\!\!\left[H\right]}}\!=\!{\frac{E\!\!\left[H\right]}{2}}\!\!\left(\!1\!+\!\mathrm{CV}{\bigl(}H{\bigr)}^{2}\right)
$$


1( ) where $E[X]$  is the probabilistic expectation of some random variable   and  $X$  $\mathrm{CV}(H)$  is the coefficient of variation of  $H,$ , a unitless measure  of the variability of  $H$  defined as model. They also found that the empirical distributions of passenger  incidence times (by time of day) had peaks just before the respec- tive average bus departure times. They hypothesized the existence  of three classes of passengers: with proportion  $q$ , passengers whose  time of incidence is causally coincident with that of a bus departure  (e.g., because they saw the approaching bus from their home or a  shop window); with proportion  $p(1-q)$ , passengers who time their  arrivals to minimize expected waiting time; and with proportion  , passengers who are randomly incident. The authors  $(1-p)(1-q)$  found that  $p$  was positively correlated with the potential reduction  in waiting time (compared with arriving randomly) that resulted  from knowledge of the timetable and of service reliability. They also  found  $p$  to be higher in the peak commuting periods rather than in  the off-peak periods, indicating more awareness of the timetable or  historical reliability, or both, by commuters.

 



Bowman and Turnquist built on the concept of aware and unaware  passengers of proportions $p$  and  $(1-p)$ , respectively. They proposed  a utility-based model to estimate $p$  and the distribution of incidence  times, and thus the mean waiting time, of aware passengers over  a given headway as a function of the headway and reliability of  bus departure times (1). They observed seven bus stops in Chicago,  Illinois, each served by a single (different) bus route, between 6:00  and  $8{:}00\;\mathrm{a.m}$ . for 5 to 10 days each. The bus routes had headways  of 5 to  $20\ \mathrm{min}$  and a range of reliabilities. The authors found that  actual average waiting time was substantially less than predicted  by the random incidence model. They estimated that  $p$  was not  statistically significantly different from 1.0, which they explain by  the fact that all observations were taken during peak commuting  times. Their model predicts that the longer the headway and the  more reliable the departures, the more peaked the distribution of  incidence times will be and the closer that peak will be to the next  scheduled departure time. This prediction demonstrates what they  refer to as a safety margin that passengers add to reduce the chance  of missing their bus when the service is known to be somewhat  unreliable. Such a safety margin can also result from unreliability in  passengers’ journeys to the public transport stop or station. Bowman  and  Turnquist conclude from their model that the random incidence  model underestimates the waiting time benefits of improving reli- ability and overestimates the waiting time benefits of increasing ser- vice frequency. This is because as reliability increases passengers  can better predict departure times and so can time their incidence to  decrease their waiting time.

Furth and Muller study the issue in a theoretical context and gener- ally agree with the above findings (2). They are primarily concerned  with the use of data from automatic vehicle-tracking systems to assess  the impacts of reliability on passenger incidence behavior and wait- ing times. They propose that passengers will react to unreliability by  departing earlier than they would with reliable services. Randomly  incident unaware passengers will experience unreliability as a more  dispersed distribution of headways and simply allocate additional  time to their trip plan to improve the chance of arriving at their des- tination on time. Aware passengers, whose incidence is not entirely  random, will react by timing their incidence somewhat earlier than  the scheduled departure time to increase their chance of catching the  desired service. The authors characterize these  reactions as the costs  of unreliability.

Luethi et al. continued with the analysis of manually collected  data on actual passenger behavior (6). They use the language  of probability to describe two classes of passengers. The first is   timetable-dependent passengers (i.e., the aware passengers), whose  incidence behavior is affected by awareness (possibly gained  through their own experience with the service) of the timetable or  service reliability, or both. The second class is timetable-independent  passengers, whose incidence behavior is random and so does not  reflect any such awareness (whether or not they have it). The lan- guage of timetable dependency is adopted for the balance of this  paper to describe the randomness of passenger incidence behavior,  regardless of what exactly is driving the behavior on the part of  the passengers. This language is preferred because it expresses the  probabilistic association between two observed variables (incidence  times and scheduled departure times) rather than some unobserved  passenger state of mind. Luethi et al. observed passenger incidence  during morning and evening peak hours and midday off-peak hours  at 28 bus, tram, and commuter rail stations in and around Zurich,  Switzerland, with headways of 2.33 to  $30\;\mathrm{min}$ . To avoid ambiguity,  they limited their station selection to nonterminal, noninterchange  stations served by a single route with constant headways over the  period of observation. The authors observed that a substantial share  of passengers appear to be timetable dependent for headways as low  as  $5\;\mathrm{min}$ . They proposed distributions of passenger incidence times  over a given headway to be the weighted superposition of two dis- tributions: one for timetable-independent passengers, with weight   distributed uniformly over the headway, and the other for  $1-p,$   timetable-dependent passengers, with weight  $p,$  distributed accord- ing to a Johnson SB distribution. The authors found the fit of their  distribution to the observed data to be statistically significant. This  distribution is parameterized in terms of the headway, but not in  terms of the reliability of the service. Estimations of their model  yield values of $p$  for the different time periods of the day, which they  find to be highest in the morning peak and lowest in the off-peak  period, supporting the conclusion that the incidence of commuters  is more timetable dependent than the incidence of noncommuters.



Csikos and Currie studied this phenomenon, first cross-sectionally  and then longitudinally, using data from the AFC system of the heavy  rail network in Melbourne, Australia (7, 8). In their first study they  used 4 weeks’ worth of data from 07:30 to 15:00, but limited them- selves to analyzing seven particular stations out of 209, for a total  of 38,000 observations over approximately  $^{1,470\mathrm{~h~}}$ . The stations  were, as in the other studies, selected to avoid ambiguity regard- ing which scheduled service each passenger intended to use. They  also obtained high-level data about the aggregate reliability (6-min  terminal on-time performance) of the train lines serving the selected  stations. Their findings generally confirm those of the other studies.  In their second study, they used the same 4-week data set, but tracked  individual ticket holders over time to study the consistency of behav- ior. They focused on the 15,000 trips made between 06:00 and 10:00  by 1,043 individual passengers who, as morning commuters, are  expected to exhibit the most consistent behavior patterns. They  characterized the passengers by the times of incidence and the offset  times until the next scheduled departure. They classified passengers  into four distinct archetypes exhibiting various levels of consistency  in these two variables, finding roughly equal numbers of passengers  in each category. On one end of the spectrum are clockwork-like  passengers who exhibit consistent behavior that often minimizes  their waiting time with respect to the schedule. On the other end,  largely random passengers have very little consistency with respect  to waiting time, exhibiting largely timetable-independent behavior.  The authors’ overall conclusion in this work is one of heterogeneity  in passenger behavior, even under homogeneous conditions (i.e., at  the same station at the same time of day served by the same line).

As a brief summary of the six studies, passenger incidence  behavior has been characterized primarily in terms of how random  it appears to be with respect to the timetable and to actual vehicle  departure times. The appearance of randomness (or its lack) has  been used to indicate the degree to which passengers have and use  knowledge of the published timetable and of actual departure times.  At longer headways, passengers have more to gain by gaining and  using knowledge of the timetable; their behavior tends to be less  random, peaking somewhat before the scheduled departure time.  Passengers also appear to gain and use knowledge of the actual,  rather than scheduled, departure times. When departure times are  reliable, even if they are reliably late (or early) by a particular  amount, incidence behavior tends to be less random, with more pas- sengers being incident shortly before the reliable departure time.  When departure times are inconsistent (i.e., unreliable), passengers  have less to gain from choosing any particular time of incidence, so  their behavior tends to be more random.



### MethodoLogy

This section proposes a method for studying passenger incidence  with respect to scheduled departure times by using AFC smartcard  data and schedule-based assignment. For a given passenger jour- ney, the method depends on the concepts of attractive departure,   scheduled waiting time (SWT), and incidence headway.

An attractive departure is a departure scheduled from the passen- ger’s station of incidence that the passenger is or would have been  willing to board, regardless of how willing is defined. The use of  “attractive” is in the tradition of Spiess and Florian and Nguyen and  Pallottino in their work on hyperpaths and optimal strategies (9, 10).  They defined the attractive set of lines as the set that a passenger  is willing to board at a given location. This concept makes explicit  the possibility that some scheduled departures may not be viable  alternatives for a given passenger as a function of that passenger’s  destination and of the subsequent itinerary of those departures. For  example, on a line with a trunk and branches, passengers bound for  one of the branches may experience longer headways than those  traveling only on the trunk.

SWT is the time the passenger should have to wait according  to the schedule, given his or her time of incidence and attractive  departures. SWT is defined as the length of time between passenger  incidence and the next attractive departure.

Incidence headway is the (scheduled) headway applicable to  the passenger given his or her time of incidence and set of attrac- tive departures. Incidence headway is defined as the length of time  between the last attractive departure before the time of incidence  and the next such departure after the time of incidence.

The above six studies of passenger incidence all selected places  and times of observation so as to avoid ambiguity with respect to  each passenger’s attractive departures. They minimized the mea- surement of SWT and incidence headways by selecting stations  served by only a single service pattern and, in some cases, with a  constant headway. While this may be sufficient for modeling the  behavior of certain group of passengers, it is clearly inadequate  for understanding behavior across an entire network. In many real- world public transport networks, the largest numbers of passengers  are incident at large stations or terminals that provide access to   heterogeneous services.

In the case of the London Overground, this incidence is most  problematic on the North London Line (NLL). Consider, for exam- ple, passengers incident to the NLL at Stratford, one of the Over- ground’s busiest stations. In 2008 peak-period timetables, the NLL  was running a mostly (but not perfectly) regular 15-min (i.e., four  trains per hour) service all day from Stratford to the end of the NLL  at Richmond. This service was augmented with occasional irregular  services, including a shuttle that ran only as far as Camden Road,  and one special that ran on the NLL to Willesden Junction but then  on the West London Line to Clapham Junction. It is not immediately  obvious which of these services would be attractive to a given pas- senger at Stratford, and thus not clear what incidence headway each  passenger would experience. Earlier literature avoided this issue by  avoiding stations such as Stratford altogether.



The method proposed here is designed as a tool to support the  study of passenger incidence behavior in general (i.e., by including  locations with heterogeneous services) and to facilitate its applica- tion in practical transit management. The method does so by estimat- ing SWT and incidence headway automatically from the integration  of published timetables with disaggregate AFC  passenger journey  data via schedule-based assignment.

Schedule-based assignment depends on a run-based model  of public transport supply, which is very similar to the line-based  model of supply, but unfolded in the temporal dimension  $(l l)$ . In  such a model, each individual scheduled or actual run (or trip) of  the public transport service is represented individually by its own  subgraph. In the subgraph for a given run, the nodes represent the  arrival, departure, or transit of that run at a specific location at a  specific time. The links represent travel (or dwelling) on that run  between specific points in time and space. The combination of the  subgraphs of all runs is referred to as the service subgraph. Demand  is also modeled with temporal as well as spatial dimensions in the  demand subgraph. Nodes in this subgraph represent centroids of  demand in time (according to user departure and arrival times)  and space (according to the physical network). The access–egress  subgraph joins the service and demand subgraphs with boarding  and alighting links. The union of these three subgraphs is referred  to as the diachronic graph representation. One benefit of such a  representation is that shortest travel time paths can be found via  standard shortest-path network algorithms such as Bellman-Ford or   Dijkstra’s (12).

In this paper it is assumed that for a given origin, destination, and  time of incidence, all passengers plan to use the single schedule- based path (i.e., set of scheduled services) through the network that  minimizes total travel time. Additionally, it is assumed that passen- gers plan itineraries to minimize the number of total boardings up  to the point at which total travel time is not increased (e.g., in the  trunk-and-branch example, branch-bound passengers won’t board  a train bound for the wrong branch just to get to the end of the  trunk and transfer to the correct branch). These assumptions are  necessarily a simplification of the true behaviors and perceptions of  passengers. The degree to which the assumptions hold is a function  of the attributes of the particular network to which they are applied  and of the behavioral preferences of the passengers in question.  These assumptions are sufficient to determine, for each passenger  journey, the attractive departures before and subsequent to the time  of incidence. SWT and incidence headway can be determined once  the times of these two departures are known.

### aLgorithM and iMPLeMentation

For a given passenger journey on a given public transport network, let  scheduled waiting time for given journey, $\mathrm{SWT}=$   incidence headway for given journey, $H_{I}=$    of passenger incidence for given  $I=\mathrm{time}$  journey,   location of incidence of journey in ques- $L_{O}=$  tion (i.e., the origin),  destination of that journey, $L_{D}=$   time of last attractive departure before  $I,$  $D_{\mathrm{prior}}=$   time of first attractive departure after  $D_{\mathrm{next}}=$ I,   maximum normal headway (i.e., time  $H_{\mathrm{max}}=$  between any two successive departures  in same direction from same location) on  network,   normal headway on network, $H_{\mathrm{min}}=\mathrm{minimum}$   Path(from, to, time)  $=$  function that finds shortest weighted travel  time path from location from to location  to with departure time strictly greater than  time, with all travel time weights equal to  1 except for transfer or boarding penalty  that is positive but less than  $H_{\mathrm{min}},$  and  Departure(path)  $=$  function that returns scheduled departure  time of path path.

          

         

        

       

      

     

    

   

  

 



Equation Box 1 illustrates the algorithm to find $H_{I}$  and SWT for the  journey in question under the above assumptions. Lines 1 through 3  accomplish the simple task of finding the next attractive departure  and thus determining SWT. Lines 4 through 9 search backward in  time in increments of  $H_{\mathrm{min}}$  until either a new attractive departure   time  $d$  is found or the time has been moved by more than  $H_{\mathrm{max}}$ .  $H_{\mathrm{min}}$   is the largest step possible such that the search will never skip over  a possible attractive departure. In theory, the algorithm could use  the timetable to determine the next departure time in this backward  search process rather than blindly stepping in increments of  $H_{\mathrm{min}}$ .  However, the algorithm is unaware of particular departure times  since the Path(  ) function encapsulates all knowledge of the timetable  itself. This particular algorithmic design is motivated primarily by  implementation concerns that are discussed in the following section.

A 272-line Perl script was written to implement the algorithm for  individual Oyster passenger journey records. The Path(  ) function 

EQUATION BOX 1  Algorithm to Find SWT and Incidence Headway  for a Given  Passenger Journey

encapsulates the complexity of conducting a schedule-based assign- ment for a single passenger trip. The Path(  ) function will either find  the earlier attractive departure time  $D_{\mathrm{prior}}$  or determine that there is  no prior attractive departure in at most  $H_{\mathrm{max}}/H_{\mathrm{min}}$  steps. If the Path(  )  function does find  $D_{\mathrm{prior}},$  the function uses that result to determine  . Embedded in this function is the algorithm for finding the short- $H_{I}$  est weighted travel time paths in a schedule-based network. The  travel time weightings are such as to enforce the assumption that  passengers minimize the number of boardings (without affecting  total travel time). A robust implementation of Path(  ) function using  a modified Dijkstra’s algorithm is available in the free open-source  software library Graphserver (13). Graphserver reads timetables in  the widely used general transit feed specification (14). This speci- fication was defined by Google to facilitate the transfer of public  transport schedules from operators to Google to power its own web- based journey planning software. The specification has become a de  facto standard for public distribution of public transport timetables.  Unfortunately, London Overground timetables do not come in gen- eral transit feed specification format, so another 302-line Perl script  was written to do the transformation (15). This algorithm improves  on the previous approaches to finding SWT and $H_{I}$  by considering the  timetable in the context of each individual journey. The origin, des- tination, and incidence time of each individual journey  determine  which departures will be attractive.

### APPLICATION TO LONDON  OVERGROUND NETWORK

This section applies the proposed methodology to examine pas- senger incidence behavior on the London Overground network by  using a large sample of passenger journey data from the Oyster  smartcard ticketing system.

Figure 1 shows a schematic map of the rail services for Trans- port for London (TfL) as of spring 2010, including London Under- ground, Dockland Light Rail, and Overground. The Overground  network is for the most part circumferential, primarily orbiting  London to the north and west, with the majority of stations in Zones  2 and 3. The Overground is very much part of the integrated net- work of TfL services, with 19 of its stations offering interchanges  to London Underground or Dockland Light Rail services. In 2010,  the Overground ran 407 scheduled weekday train trips with 27 units  of rolling stock (A. Brimbacombe, personal communication, 2010).



Services on the Overground are divided into four service patterns  (Table 1). The core of this network is the NLL, which runs  $28\ \mathrm{km}$   between Stratford in the northeast of London and Richmond in the  southwest, connecting to every other Overground service and to  numerous other TfL and National Rail services along the way. The  NLL is by far the busiest Overground line, with the most frequent ser- vice and an estimated  $58\%$  of all Overground boardings (C. Smales, 



Note: tph = trains per hour.

personal communication, 2010). The NLL runs four (end-to-end)  trains per hour over most of the day, with some segments receiving  six trains per hour during peak periods. The other Overground lines  run at lower frequencies (three trains per hour during the peak  periods  and two to three trains per hour during other periods).

### data

On November 11, 2007, overall management and revenue respon- sibility for this set of services was transferred to TfL from the   Silverlink, which held the prior National Rail franchise. At that  time, the network was rebranded as the Overground and became  fully Oyster enabled. Oyster is TfL’s AFC smartcard system. London’s  fare policy and technologies require most Oyster users to validate  their cards on all entries and exits to the system. The centralized  computer system archives these Oyster entry and exit transactions,  including their location, time stamp, and Oyster ID, in an easily  accessible database. As a result, disaggregate Oyster journey data  are cheap to gather in large volumes and provide a prime source of  data on individual passenger journeys.

The data analyzed here are a  $100\%$  sample of all Oyster journeys  between all pairs of London Overground stations for the 52 business  days from March 31, 2008, through June 10, 2008, inclusive. Pub- lic timetables were obtained for the Overground network. The data  set was filtered to include only those journeys for which it can be  assumed with relative certainty that the passenger in question used  only Overground services; that is, this data set does not include jour- neys that interchanged to the Overground but with initial Oyster val- idations elsewhere in the railway system. The data set also excludes  journeys initially incident to the Overground but interchanging to  other railway services. Journeys with interchanges to or from buses  will be included here, since the Oyster system effectively separates  the recording of bus and rail journeys. Passengers transferring from  buses to London Overground may exhibit a different incidence  behavior because they do not have control over the time when the  bus arrives at the transfer stops. Their incidences at London Over- ground stations are only a function of the timetable coordination  between bus and Overground services, if there is any such coordina- tion. In general these journeys may demonstrate random incidence  behavior. The resulting data set contains nearly 1,670,000 journeys  from 54 stations on 1,442 origin– destination pairs made by more  than 290,000 passengers. The data set constitutes approximately  53,000 station-hours of observation of  passenger incidence to   Overground services.

The methodology described in the previous section was applied  to each Oyster journey in the data set: the transaction time of the  Oyster entry was taken as the time of incidence, and the origin and  destination of the journey were used along with the timetable to  estimate which services the journey was incident to. This processing  took some number of hours for the entire data set, but it was fully  automated. A large set of observations was obtained for which the  following data were measured or estimated: • Date and time of incidence, • Incidence headway, • SWT, • Location (i.e., station) of incidence (i.e., the journey’s origin), • Journey’s destination, and • Overground line to which the passenger was incident.

 



For journeys that required an interchange within the Overground  system (of which there are relatively few), the above data were   measured or estimated for only the first incidence event.

As a point of validation, Figure 2 plots distributions of incidence  headway for passengers on the Gospel Oak to Barking (GOB) and  North London Lines. The findings are consistent with expectations.  On the NLL, the mode of all the distributions is  $15~\mathrm{min}$ , reflect- ing the core service. The distribution is more concentrated during  the interpeak period, when there are no scheduled shuttles or spe- cials. The opposite is true on the GOB, which runs a regular 20-min   service in the peak periods but transitions to and from a 30-min  service in the interpeak period. The morning peak distribution is  somewhat more dispersed than that of the evening peak because the  morning peak includes a transition from 30-min headways in the  early morning.

### results

Consistent with the reviewed studies of passenger incidence behav- ior, the first results of interest are distributions of passenger inci- dence time over a given headway. Figure 3 plots these distributions  for the London Overground network by line and by time period.  In this plot, incidence times are normalized by the incidence head- way because different passengers experience different incidence  headways.

It is clear from Figure 3 that passenger incidence behavior, with  respect to published timetables, varies spatially and temporally across  the Overground network. Passenger incidence on the GOB and   Watford DC lines is much more peaked (i.e., timetable-dependent),  each with 20-min headways, than on the NLL, with 7- to 15-min  headways, during any time period. Also, the NLL is acknowledged by  Overground management to have the most serious reliability problems  (O. Bratton, personal communication, 2010). These variations are con- sistent with the literature that passenger incidence is more timetable- dependent with a more peaked distribution for longer headways and  for more reliable services.

Also consistent with the literature is that, for all lines, the distri- bution is more peaked in the morning peak period than in the eve- ning peak or midday interpeak periods. It appears that Overground  commuters in the morning peak period, more likely to have knowl- edge of the timetable and the service and more sensitive to time  saving, exhibit less random incidence behavior than do passengers  in other time periods, despite the shorter headways and less reliable  service found in the morning peak. The distribution in the late-night  period is also very peaked because of longer headways and possibly  higher sensitivity to time.

The peaks of all of the distributions are somewhat before the very  end of the headway, indicating some type of safety margin or wait- ing time–minimization behavior on the part of passengers. Many of  the distributions have small spikes at the beginning of the headway,  indicating possible late running awareness among some passengers.  While such awareness may in fact be found on the Overground, it is  also possible that it is the passengers themselves who are running late.  They may be planning to take a train scheduled to depart at a certain  time but, because of uncontrollable circumstances or their own poor  planning, arrive at the station shortly after that departure time.

Figure 4 shows the mean waiting time, for each line and time period,  under two models of passenger behavior and train operations: random  incidence versus timetable-dependent incidence. In the first case, it is  assumed that all passengers are randomly incident to constant head- way services, so the mean waiting time is calculated as half the mean  incidence headway. In the second case, no behavioral assumption is  made and all trains are assumed to run as per the timetable, so the  observed mean waiting time is calculated as the mean SWT. Figure 4  thus indicates the effects of the observed incidence behaviors  (compared with random incidence) on passenger waiting times.

On the NLL, the relatively slight skew in the incidence distribu- tions translates into relatively small impacts on waiting time. In the  morning peak, timetable dependence decreases waiting time by $7.2\%$   from 6.82 to $6.33\,\mathrm{min}$  (about 30 s). In the interpeak and evening peak  periods, the reductions are only  $0.2\%$  and $2.2\%$ , respectively. On the  GOB, however, the implications of timetable dependence are substan- tial. In the morning peak, the waiting time decreases by $29\%$  (3.1 min)  from 10.5 to  $7.4\,\mathrm{min}$ . In the interpeak and evening peak periods, the  reductions are  $20.4\%$  and  $17.5\%$ , respectively.

Transit assignment models often make an assumption that no ser- vice would be assigned a mean passenger waiting time over a certain  threshold, say,  $10\;\mathrm{min}$  in the London Overground case. The results  of this section lend support to this assumption. Regular weekday  headways on the Overground network go as high as  $30\,\mathrm{min}$ , but the  observed mean SWT is above 11 min in only two cases (on the West  London Line during the interpeak and evening time periods) and is  never above  $12.1\;\mathrm{min}$ .

### discussion and concLusions

This paper developed a methodology to relate disaggregate AFC  journey data to published timetables for the purpose of studying  passenger incidence behavior and applied this methodology to the  London Overground (16).

The following conclusions are drawn about the methodology  developed here. First, it can be used to study passenger incidence  behavior using large samples of disaggregate journey data from  AFC systems such as the Oyster smartcard system. Second, the  methodology can for each passenger journey estimate SWT and  incidence headway under heterogeneous conditions. Finally, this  methodology is able to efficiently process millions of data records  and can be implemented using open standard timetable formats and  free software tools.

As a limitation the proposed method hinges on the schedule- based assignment, for which there are unsolved methodological  issues such as station congestion or train crowding. The London  Overground is a relatively simple network (i.e., compared with the  London Underground), so that the methodology would not be sub- ject to those issues. As the inner workings of schedule-based assign- ment improve, the method of this paper will continue to become  more broadly applicable.

With respect to the London Overground, the following conclu- sions can be drawn. Generally, passenger incidence behavior varies  across the network and across times of day, and the differences reflect  different headways, reliability of service, and time of day (indicating  different knowledge of the network and sensitivity to time); these  findings reflect the findings in the literature to date. Specifically, inci- dence appears to be much less timetable dependent on the NLL than  on the other Overground lines because of shorter headways and less  reliable service on this line as compared with others. The method  and results of the paper also provide one service quality measure,  SWT, for the entire Overground network. On the GOB, Watford DC,  and West London lines, where incidence behavior is more timetable- dependent, passengers reduce their mean SWT by more than  $3\;\mathrm{min}$ ,  or up to  $30\%$ , during daytime hours compared with random inci- dence behavior. On the NLL, such reductions are much smaller, in  some cases nearly zero, in both relative and absolute terms.



The method developed in this paper can support further study  of passenger incidence behavior in a more systematic way; that is,  researchers no longer need to select a special subset of stations with  only one service pattern. The work of Bowman and Turnquist has  been influential in shaping the understanding of the relationships  between headway, reliability, passenger behavior, and waiting time  (1). Their work should be updated by using the method of this paper so  that large samples of passenger data across heterogeneous networks  can be easily analyzed. The London Overground network represents  an ideal opportunity to conduct such a study because its passengers  can clearly be studied via Oyster data, and its trains are tracked by a  computerized signaling system. After the East  London Line opened,  the network has headways ranging from 5 to  $30\ \mathrm{min}$  during most  hours of the day, thus providing a good range of variation.

Another application is to study the impact of real-time informa- tion. Many strides have been made toward informing passengers in  real time about the status of public transport services. Such infor- mation is now often distributed via in-station signs and announce- ments, as well as over the Internet to passengers’ computers and,  more importantly, mobile devices. It is crucial to advance the  understanding of passenger incidence to include the effects of real- time information. Such an advance in understanding will require  careful thinking and research designs, but it should be able to take  advantage of the methodology developed here. For example, actual  recorded train arrival and departure times (rather than published  timetables) could be used in the schedule-based assignment.

### acknowLedgMents

The authors thank Nigel Wilson for cosupervising the research proj- ect and providing critical input to the master’s thesis on which this  paper is based. The authors also thank Transport for London for pro- viding the financial and institutional support for this research, the  opportunity to work in the agency, and access to the data used in this  paper and specifically thank Oliver Bratton of London Overground  Rail Operations Ltd for insight into the issues and the dynamics of  the London Overground network.



### references

 1. Bowman, L. A., and M. A. Turnquist. Service Frequency, Schedule  Reliability, and Passenger Wait Times at Transit Stops. Transportation  Research Part A, Vol. 15, No. 6, 1981, pp. 465–471.

 2. Furth, P. G., and T. H. J. Muller. Service Reliability and Hidden Wait- ing Time. In Transportation Research Record 1955, TRB, National  Research Council, Washington, D.C., 2006, pp. 79–87.

 3. Larson, R. C., and A. R. Odoni. Urban Operations Research, 2nd ed.  Dynamic Ideas, Belmont, Mass., 2007.

 4. Osuna, E. E., and G. F. Newell. Control Strategies for an Idealized Pub- lic Transportation System. Transportation Science, Vol. 6, No. 1, 1972,  pp. 52–71.

 5. Jolliffe, J. K., and T. P. Hutchinson. A Behavioral Explanation of the  Association Between Bus and Passenger Arrivals at a Bus Stop. Trans- portation Science, No. 9, 1975, pp. 248–282.

 6. Luethi, M., U. A. Weidmann, and A. Nash. Passenger Arrival Rates at  Public Transport Stations. Presented at 86th Annual Meeting of the  Transportation Research Board, Washington, D.C., 2007.

 7. Csikos, D. R., and G. Currie. Investigating Consistency in Transit Pas- senger Arrivals: Insights from Longitudinal Automated Fare Collection  Data. In Transportation Research Record: Journal of the Transporta- tion Research Board, No. 2042, Transportation Research Board of the  National Academies, Washington, D.C., 2008, pp. 12–19.

 8. Csikos, D. R., and G. Currie. The Impacts of Transit Reliability on Wait  Time: Insights from Automated Fare Collection System Data. Pre- sented at 86th Annual Meeting of the Transportation Research Board,   Washington, D.C., 2007.

 9. Spiess, H., and M. Florian. Optimal Strategies: A New Assignment  Model for Transit Networks. Transportation Research Part B, Vol. 23,  No. 2, 1989, pp. 83–102.

10. Nguyen, S., and S. Pallottino. Equilibrium Traffic Assignment for  Large-Scale Transit Networks. European Journal of Operations Research,  Vol. 37, 1988, pp. 176–186.

11. Nuzzolo, A., and U. Crisalli. The Schedule-Based Approach in Dynamic  Transit Modeling: A General Overview. In Schedule-Based Dynamic  Transit Modeling: Theory and Applications (N. H. M. Wilson and  A. Nuzzolo, eds.). Operations Research/Computer Science Interface  Series, Kluwer Academic Publishers, 2004, pp. 1–24.

12. Bertsimas, D., and J. N. Tsitsiklis. Introduction to Linear Optimization.  Dynamic Ideas, Belmont, Mass., 1997.

13. Graphserver. Graphserver: The Open-Source Multi-Modal Trip Planner.  http://bmander.github.com/graphserver/. Accessed 2009.

14. Google. General Transit Feed Specification Reference. http://code. google.com/transit/spec/transit_feed_specification.html. Accessed  Nov. 13, 2009.

15. Perl.org. The Perl Programming Language. http://www.perl.org/.  Accessed 2010.

16. Frumin, M. Automatic Data for Applied Railway Management: Passen- ger Demand, Service Quality Measurement, and Tactical Planning on  the London Overground Network. MS thesis. Massachusetts Institute of  Technology, Cambridge, Mass., 2010.

The Transit Management and Performance Committee peer-reviewed this paper.
# Analyzing Passenger Incidence Behavior   in Heterogeneous Transit Services   Using Smartcard Data and   Schedule-Based Assignment

Michael Frumin and Jinhua Zhao

Passenger incidence (station arrival) behavior has been studied primar- ily to understand how changes to a transit service will affect passenger  waiting times. The impact of one intervention (e.g., increasing frequency)  could be overestimated when compared with another (e.g., improving  reliability), depending on the assumption of incidence behavior. Under- standing passenger incidence allows management decisions to be based on  realistic behavioral assumptions. Earlier studies on passenger incidence  chose their data samples from stations with a single service pattern such  that the linking of passengers to services was straightforward. This choice  of data samples simplifies the analysis but heavily limits the stations that  can be studied. In any moderately complex network, many stations may  have more than one service pattern. This limitation prevents the method  from being systematically applied to the whole network and constrains its  use in practice. This paper considers incidence behavior in stations with  heterogeneous services and proposes a method for estimating incidence  headway and waiting time by integrating disaggregate smartcard data  with published timetables using schedule-based assignment. This method  is applied to stations in the entire London Overground to demonstrate  its practicality; incidence behavior varies across the network and across  times of day and reflects headways and reliability. Incidence is much  less timetable-dependent on the North London Line than on the other  lines because of shorter headways and poorer reliability. Where incidence  is timetable-dependent, passengers reduce their mean scheduled waiting  time by more than 3 min compared with random incidence.

This paper is concerned with passenger incidence behavior, which  is defined as the act or event of a person’s being incident to a public  transport service with intent to use that service. A lexical convention  is established here to avoid ambiguity between passenger incidence  to public transport services and arrival at certain destinations after  using public transport services.

Passenger incidence behavior has been studied primarily for the  sake of understanding how changes to a public transport service will  affect passenger waiting times. It is also of interest because it affects  the relationship between the departure times of public transport  vehicles and the passenger loads on those vehicles. Understanding  passenger incidence behaviors allows management interventions to  be based on realistic behavioral assumptions. As pointed out in one  of the seminal investigations on the topic by Bowman and Turnquist,  the effects on passenger waiting time of one particular intervention  (e.g., increasing frequency) could be overestimated compared with a  different type of intervention (e.g., improving reliability),  depending  on which assumptions are made about incidence behavior (1).



Earlier studies on passenger incidence behavior chose their data  samples from stations or stops with a single service pattern such  that the linking of passengers to scheduled or actual services was  straightforward. This choice of data samples simplifies the analysis  and may suffice for an academic study. But such simplification also  heavily limits the stations or stops that can be studied in the transit  network. In any moderately complex network, many stations may  have more than one service pattern. This limitation prevents the  method from being systematically applied to the entire network and  therefore limits its use in transit management practice. In environ- ments in which passengers at a given station have a choice of ser- vices, a more sophisticated approach is needed to study passenger  incidence behavior.

This paper is concerned with the relationship between the times  of passenger incidence and published timetables in a rail network  with heterogeneous service patterns. It proposes a method to study  this relationship by integrating disaggregate passenger journey data  from automatic fare collection (AFC) smartcard systems with pub- lished timetables using schedule-based assignment. The purpose of  this paper is twofold: first, to develop a method that contributes to  the study of passenger incidence behavior across a railway network  with heterogeneous service patterns and frequencies using pub- lished timetables and AFC data; and second, to apply this method  to the London Overground to demonstrate its practicality and shed  light on the incidence behavior of its passengers.

### Literature

Previous research has identified a rich set of passenger incidence behav- iors and related them to certain aspects of public transport services.  Using manually and automatically collected data sources, research  has investigated the diversity of the behavior cross- sectionally and the  consistency of such behaviors longitudinally over time. It has been  found that the randomness of passenger incidence behavior is highly  dependent on the service headway and the reliability of the departure  time of the service to which passengers are incident.



After briefly introducing the random incidence model, which is  often assumed to hold at short headways, the balance of this section  reviews six studies of passenger incidence behavior that are moti- vated by understanding the relationships between service headway,  service reliability, passenger incidence behavior, and passenger  waiting time in a more nuanced fashion than is embedded in the  random incidence assumption (2). Three of these studies depend on  manually collected data, two studies use data from AFC systems,  and one study analyzes the issue purely theoretically. These studies  reveal much about passenger incidence behavior, but all are found  to be limited in their general applicability by the methods with  which they collect information about passengers and the services  those passengers intend to use.

### random Passenger incidence Behavior

One characterization of passenger incidence behavior is that of ran- dom incidence (3). The key assumption underlying the random inci- dence model is that the process of passenger arrivals to the public  transport service is independent from the vehicle departure process  of the service. This implies that passengers become incident to the  service at a random time, and thus the instantaneous rate of passen- ger arrivals to the service is uniform over a given period of time. Let  W and $H$  be random variables representing passenger waiting times  and service headways, respectively. Under the random incidence  assumption and the assumption that vehicle capacity is not a binding  constraint, a classic result of transportation science is that

## 
$$
E{\bigl(}W{\bigr)}\!=\!{\frac{E\!\!\left[H^{2}\right]}{2E\!\!\left[H\right]}}\!=\!{\frac{E\!\!\left[H\right]}{2}}\!\!\left(\!1\!+\!\mathrm{CV}{\bigl(}H{\bigr)}^{2}\right)
$$


1( ) where $E[X]$  is the probabilistic expectation of some random variable   and  $X$  $\mathrm{CV}(H)$  is the coefficient of variation of  $H,$ , a unitless measure  of the variability of  $H$  defined as model. They also found that the empirical distributions of passenger  incidence times (by time of day) had peaks just before the respec- tive average bus departure times. They hypothesized the existence  of three classes of passengers: with proportion  $q$ , passengers whose  time of incidence is causally coincident with that of a bus departure  (e.g., because they saw the approaching bus from their home or a  shop window); with proportion  $p(1-q)$ , passengers who time their  arrivals to minimize expected waiting time; and with proportion  , passengers who are randomly incident. The authors  $(1-p)(1-q)$  found that  $p$  was positively correlated with the potential reduction  in waiting time (compared with arriving randomly) that resulted  from knowledge of the timetable and of service reliability. They also  found  $p$  to be higher in the peak commuting periods rather than in  the off-peak periods, indicating more awareness of the timetable or  historical reliability, or both, by commuters.

 



Bowman and Turnquist built on the concept of aware and unaware  passengers of proportions $p$  and  $(1-p)$ , respectively. They proposed  a utility-based model to estimate $p$  and the distribution of incidence  times, and thus the mean waiting time, of aware passengers over  a given headway as a function of the headway and reliability of  bus departure times (1). They observed seven bus stops in Chicago,  Illinois, each served by a single (different) bus route, between 6:00  and  $8{:}00\;\mathrm{a.m}$ . for 5 to 10 days each. The bus routes had headways  of 5 to  $20\ \mathrm{min}$  and a range of reliabilities. The authors found that  actual average waiting time was substantially less than predicted  by the random incidence model. They estimated that  $p$  was not  statistically significantly different from 1.0, which they explain by  the fact that all observations were taken during peak commuting  times. Their model predicts that the longer the headway and the  more reliable the departures, the more peaked the distribution of  incidence times will be and the closer that peak will be to the next  scheduled departure time. This prediction demonstrates what they  refer to as a safety margin that passengers add to reduce the chance  of missing their bus when the service is known to be somewhat  unreliable. Such a safety margin can also result from unreliability in  passengers’ journeys to the public transport stop or station. Bowman  and  Turnquist conclude from their model that the random incidence  model underestimates the waiting time benefits of improving reli- ability and overestimates the waiting time benefits of increasing ser- vice frequency. This is because as reliability increases passengers  can better predict departure times and so can time their incidence to  decrease their waiting time.

Furth and Muller study the issue in a theoretical context and gener- ally agree with the above findings (2). They are primarily concerned  with the use of data from automatic vehicle-tracking systems to assess  the impacts of reliability on passenger incidence behavior and wait- ing times. They propose that passengers will react to unreliability by  departing earlier than they would with reliable services. Randomly  incident unaware passengers will experience unreliability as a more  dispersed distribution of headways and simply allocate additional  time to their trip plan to improve the chance of arriving at their des- tination on time. Aware passengers, whose incidence is not entirely  random, will react by timing their incidence somewhat earlier than  the scheduled departure time to increase their chance of catching the  desired service. The authors characterize these  reactions as the costs  of unreliability.

Luethi et al. continued with the analysis of manually collected  data on actual passenger behavior (6). They use the language  of probability to describe two classes of passengers. The first is   timetable-dependent passengers (i.e., the aware passengers), whose  incidence behavior is affected by awareness (possibly gained  through their own experience with the service) of the timetable or  service reliability, or both. The second class is timetable-independent  passengers, whose incidence behavior is random and so does not  reflect any such awareness (whether or not they have it). The lan- guage of timetable dependency is adopted for the balance of this  paper to describe the randomness of passenger incidence behavior,  regardless of what exactly is driving the behavior on the part of  the passengers. This language is preferred because it expresses the  probabilistic association between two observed variables (incidence  times and scheduled departure times) rather than some unobserved  passenger state of mind. Luethi et al. observed passenger incidence  during morning and evening peak hours and midday off-peak hours  at 28 bus, tram, and commuter rail stations in and around Zurich,  Switzerland, with headways of 2.33 to  $30\;\mathrm{min}$ . To avoid ambiguity,  they limited their station selection to nonterminal, noninterchange  stations served by a single route with constant headways over the  period of observation. The authors observed that a substantial share  of passengers appear to be timetable dependent for headways as low  as  $5\;\mathrm{min}$ . They proposed distributions of passenger incidence times  over a given headway to be the weighted superposition of two dis- tributions: one for timetable-independent passengers, with weight   distributed uniformly over the headway, and the other for  $1-p,$   timetable-dependent passengers, with weight  $p,$  distributed accord- ing to a Johnson SB distribution. The authors found the fit of their  distribution to the observed data to be statistically significant. This  distribution is parameterized in terms of the headway, but not in  terms of the reliability of the service. Estimations of their model  yield values of $p$  for the different time periods of the day, which they  find to be highest in the morning peak and lowest in the off-peak  period, supporting the conclusion that the incidence of commuters  is more timetable dependent than the incidence of noncommuters.



Csikos and Currie studied this phenomenon, first cross-sectionally  and then longitudinally, using data from the AFC system of the heavy  rail network in Melbourne, Australia (7, 8). In their first study they  used 4 weeks’ worth of data from 07:30 to 15:00, but limited them- selves to analyzing seven particular stations out of 209, for a total  of 38,000 observations over approximately  $^{1,470\mathrm{~h~}}$ . The stations  were, as in the other studies, selected to avoid ambiguity regard- ing which scheduled service each passenger intended to use. They  also obtained high-level data about the aggregate reliability (6-min  terminal on-time performance) of the train lines serving the selected  stations. Their findings generally confirm those of the other studies.  In their second study, they used the same 4-week data set, but tracked  individual ticket holders over time to study the consistency of behav- ior. They focused on the 15,000 trips made between 06:00 and 10:00  by 1,043 individual passengers who, as morning commuters, are  expected to exhibit the most consistent behavior patterns. They  characterized the passengers by the times of incidence and the offset  times until the next scheduled departure. They classified passengers  into four distinct archetypes exhibiting various levels of consistency  in these two variables, finding roughly equal numbers of passengers  in each category. On one end of the spectrum are clockwork-like  passengers who exhibit consistent behavior that often minimizes  their waiting time with respect to the schedule. On the other end,  largely random passengers have very little consistency with respect  to waiting time, exhibiting largely timetable-independent behavior.  The authors’ overall conclusion in this work is one of heterogeneity  in passenger behavior, even under homogeneous conditions (i.e., at  the same station at the same time of day served by the same line).

As a brief summary of the six studies, passenger incidence  behavior has been characterized primarily in terms of how random  it appears to be with respect to the timetable and to actual vehicle  departure times. The appearance of randomness (or its lack) has  been used to indicate the degree to which passengers have and use  knowledge of the published timetable and of actual departure times.  At longer headways, passengers have more to gain by gaining and  using knowledge of the timetable; their behavior tends to be less  random, peaking somewhat before the scheduled departure time.  Passengers also appear to gain and use knowledge of the actual,  rather than scheduled, departure times. When departure times are  reliable, even if they are reliably late (or early) by a particular  amount, incidence behavior tends to be less random, with more pas- sengers being incident shortly before the reliable departure time.  When departure times are inconsistent (i.e., unreliable), passengers  have less to gain from choosing any particular time of incidence, so  their behavior tends to be more random.



### MethodoLogy

This section proposes a method for studying passenger incidence  with respect to scheduled departure times by using AFC smartcard  data and schedule-based assignment. For a given passenger jour- ney, the method depends on the concepts of attractive departure,   scheduled waiting time (SWT), and incidence headway.

An attractive departure is a departure scheduled from the passen- ger’s station of incidence that the passenger is or would have been  willing to board, regardless of how willing is defined. The use of  “attractive” is in the tradition of Spiess and Florian and Nguyen and  Pallottino in their work on hyperpaths and optimal strategies (9, 10).  They defined the attractive set of lines as the set that a passenger  is willing to board at a given location. This concept makes explicit  the possibility that some scheduled departures may not be viable  alternatives for a given passenger as a function of that passenger’s  destination and of the subsequent itinerary of those departures. For  example, on a line with a trunk and branches, passengers bound for  one of the branches may experience longer headways than those  traveling only on the trunk.

SWT is the time the passenger should have to wait according  to the schedule, given his or her time of incidence and attractive  departures. SWT is defined as the length of time between passenger  incidence and the next attractive departure.

Incidence headway is the (scheduled) headway applicable to  the passenger given his or her time of incidence and set of attrac- tive departures. Incidence headway is defined as the length of time  between the last attractive departure before the time of incidence  and the next such departure after the time of incidence.

The above six studies of passenger incidence all selected places  and times of observation so as to avoid ambiguity with respect to  each passenger’s attractive departures. They minimized the mea- surement of SWT and incidence headways by selecting stations  served by only a single service pattern and, in some cases, with a  constant headway. While this may be sufficient for modeling the  behavior of certain group of passengers, it is clearly inadequate  for understanding behavior across an entire network. In many real- world public transport networks, the largest numbers of passengers  are incident at large stations or terminals that provide access to   heterogeneous services.

In the case of the London Overground, this incidence is most  problematic on the North London Line (NLL). Consider, for exam- ple, passengers incident to the NLL at Stratford, one of the Over- ground’s busiest stations. In 2008 peak-period timetables, the NLL  was running a mostly (but not perfectly) regular 15-min (i.e., four  trains per hour) service all day from Stratford to the end of the NLL  at Richmond. This service was augmented with occasional irregular  services, including a shuttle that ran only as far as Camden Road,  and one special that ran on the NLL to Willesden Junction but then  on the West London Line to Clapham Junction. It is not immediately  obvious which of these services would be attractive to a given pas- senger at Stratford, and thus not clear what incidence headway each  passenger would experience. Earlier literature avoided this issue by  avoiding stations such as Stratford altogether.



The method proposed here is designed as a tool to support the  study of passenger incidence behavior in general (i.e., by including  locations with heterogeneous services) and to facilitate its applica- tion in practical transit management. The method does so by estimat- ing SWT and incidence headway automatically from the integration  of published timetables with disaggregate AFC  passenger journey  data via schedule-based assignment.

Schedule-based assignment depends on a run-based model  of public transport supply, which is very similar to the line-based  model of supply, but unfolded in the temporal dimension  $(l l)$ . In  such a model, each individual scheduled or actual run (or trip) of  the public transport service is represented individually by its own  subgraph. In the subgraph for a given run, the nodes represent the  arrival, departure, or transit of that run at a specific location at a  specific time. The links represent travel (or dwelling) on that run  between specific points in time and space. The combination of the  subgraphs of all runs is referred to as the service subgraph. Demand  is also modeled with temporal as well as spatial dimensions in the  demand subgraph. Nodes in this subgraph represent centroids of  demand in time (according to user departure and arrival times)  and space (according to the physical network). The access–egress  subgraph joins the service and demand subgraphs with boarding  and alighting links. The union of these three subgraphs is referred  to as the diachronic graph representation. One benefit of such a  representation is that shortest travel time paths can be found via  standard shortest-path network algorithms such as Bellman-Ford or   Dijkstra’s (12).

In this paper it is assumed that for a given origin, destination, and  time of incidence, all passengers plan to use the single schedule- based path (i.e., set of scheduled services) through the network that  minimizes total travel time. Additionally, it is assumed that passen- gers plan itineraries to minimize the number of total boardings up  to the point at which total travel time is not increased (e.g., in the  trunk-and-branch example, branch-bound passengers won’t board  a train bound for the wrong branch just to get to the end of the  trunk and transfer to the correct branch). These assumptions are  necessarily a simplification of the true behaviors and perceptions of  passengers. The degree to which the assumptions hold is a function  of the attributes of the particular network to which they are applied  and of the behavioral preferences of the passengers in question.  These assumptions are sufficient to determine, for each passenger  journey, the attractive departures before and subsequent to the time  of incidence. SWT and incidence headway can be determined once  the times of these two departures are known.

### aLgorithM and iMPLeMentation

For a given passenger journey on a given public transport network, let  scheduled waiting time for given journey, $\mathrm{SWT}=$   incidence headway for given journey, $H_{I}=$    of passenger incidence for given  $I=\mathrm{time}$  journey,   location of incidence of journey in ques- $L_{O}=$  tion (i.e., the origin),  destination of that journey, $L_{D}=$   time of last attractive departure before  $I,$  $D_{\mathrm{prior}}=$   time of first attractive departure after  $D_{\mathrm{next}}=$ I,   maximum normal headway (i.e., time  $H_{\mathrm{max}}=$  between any two successive departures  in same direction from same location) on  network,   normal headway on network, $H_{\mathrm{min}}=\mathrm{minimum}$   Path(from, to, time)  $=$  function that finds shortest weighted travel  time path from location from to location  to with departure time strictly greater than  time, with all travel time weights equal to  1 except for transfer or boarding penalty  that is positive but less than  $H_{\mathrm{min}},$  and  Departure(path)  $=$  function that returns scheduled departure  time of path path.

          

         

        

       

      

     

    

   

  

 



Equation Box 1 illustrates the algorithm to find $H_{I}$  and SWT for the  journey in question under the above assumptions. Lines 1 through 3  accomplish the simple task of finding the next attractive departure  and thus determining SWT. Lines 4 through 9 search backward in  time in increments of  $H_{\mathrm{min}}$  until either a new attractive departure   time  $d$  is found or the time has been moved by more than  $H_{\mathrm{max}}$ .  $H_{\mathrm{min}}$   is the largest step possible such that the search will never skip over  a possible attractive departure. In theory, the algorithm could use  the timetable to determine the next departure time in this backward  search process rather than blindly stepping in increments of  $H_{\mathrm{min}}$ .  However, the algorithm is unaware of particular departure times  since the Path(  ) function encapsulates all knowledge of the timetable  itself. This particular algorithmic design is motivated primarily by  implementation concerns that are discussed in the following section.

A 272-line Perl script was written to implement the algorithm for  individual Oyster passenger journey records. The Path(  ) function 

EQUATION BOX 1  Algorithm to Find SWT and Incidence Headway  for a Given  Passenger Journey

encapsulates the complexity of conducting a schedule-based assign- ment for a single passenger trip. The Path(  ) function will either find  the earlier attractive departure time  $D_{\mathrm{prior}}$  or determine that there is  no prior attractive departure in at most  $H_{\mathrm{max}}/H_{\mathrm{min}}$  steps. If the Path(  )  function does find  $D_{\mathrm{prior}},$  the function uses that result to determine  . Embedded in this function is the algorithm for finding the short- $H_{I}$  est weighted travel time paths in a schedule-based network. The  travel time weightings are such as to enforce the assumption that  passengers minimize the number of boardings (without affecting  total travel time). A robust implementation of Path(  ) function using  a modified Dijkstra’s algorithm is available in the free open-source  software library Graphserver (13). Graphserver reads timetables in  the widely used general transit feed specification (14). This speci- fication was defined by Google to facilitate the transfer of public  transport schedules from operators to Google to power its own web- based journey planning software. The specification has become a de  facto standard for public distribution of public transport timetables.  Unfortunately, London Overground timetables do not come in gen- eral transit feed specification format, so another 302-line Perl script  was written to do the transformation (15). This algorithm improves  on the previous approaches to finding SWT and $H_{I}$  by considering the  timetable in the context of each individual journey. The origin, des- tination, and incidence time of each individual journey  determine  which departures will be attractive.

### APPLICATION TO LONDON  OVERGROUND NETWORK

This section applies the proposed methodology to examine pas- senger incidence behavior on the London Overground network by  using a large sample of passenger journey data from the Oyster  smartcard ticketing system.

Figure 1 shows a schematic map of the rail services for Trans- port for London (TfL) as of spring 2010, including London Under- ground, Dockland Light Rail, and Overground. The Overground  network is for the most part circumferential, primarily orbiting  London to the north and west, with the majority of stations in Zones  2 and 3. The Overground is very much part of the integrated net- work of TfL services, with 19 of its stations offering interchanges  to London Underground or Dockland Light Rail services. In 2010,  the Overground ran 407 scheduled weekday train trips with 27 units  of rolling stock (A. Brimbacombe, personal communication, 2010).



Services on the Overground are divided into four service patterns  (Table 1). The core of this network is the NLL, which runs  $28\ \mathrm{km}$   between Stratford in the northeast of London and Richmond in the  southwest, connecting to every other Overground service and to  numerous other TfL and National Rail services along the way. The  NLL is by far the busiest Overground line, with the most frequent ser- vice and an estimated  $58\%$  of all Overground boardings (C. Smales, 



Note: tph = trains per hour.

personal communication, 2010). The NLL runs four (end-to-end)  trains per hour over most of the day, with some segments receiving  six trains per hour during peak periods. The other Overground lines  run at lower frequencies (three trains per hour during the peak  periods  and two to three trains per hour during other periods).

### data

On November 11, 2007, overall management and revenue respon- sibility for this set of services was transferred to TfL from the   Silverlink, which held the prior National Rail franchise. At that  time, the network was rebranded as the Overground and became  fully Oyster enabled. Oyster is TfL’s AFC smartcard system. London’s  fare policy and technologies require most Oyster users to validate  their cards on all entries and exits to the system. The centralized  computer system archives these Oyster entry and exit transactions,  including their location, time stamp, and Oyster ID, in an easily  accessible database. As a result, disaggregate Oyster journey data  are cheap to gather in large volumes and provide a prime source of  data on individual passenger journeys.

The data analyzed here are a  $100\%$  sample of all Oyster journeys  between all pairs of London Overground stations for the 52 business  days from March 31, 2008, through June 10, 2008, inclusive. Pub- lic timetables were obtained for the Overground network. The data  set was filtered to include only those journeys for which it can be  assumed with relative certainty that the passenger in question used  only Overground services; that is, this data set does not include jour- neys that interchanged to the Overground but with initial Oyster val- idations elsewhere in the railway system. The data set also excludes  journeys initially incident to the Overground but interchanging to  other railway services. Journeys with interchanges to or from buses  will be included here, since the Oyster system effectively separates  the recording of bus and rail journeys. Passengers transferring from  buses to London Overground may exhibit a different incidence  behavior because they do not have control over the time when the  bus arrives at the transfer stops. Their incidences at London Over- ground stations are only a function of the timetable coordination  between bus and Overground services, if there is any such coordina- tion. In general these journeys may demonstrate random incidence  behavior. The resulting data set contains nearly 1,670,000 journeys  from 54 stations on 1,442 origin– destination pairs made by more  than 290,000 passengers. The data set constitutes approximately  53,000 station-hours of observation of  passenger incidence to   Overground services.

The methodology described in the previous section was applied  to each Oyster journey in the data set: the transaction time of the  Oyster entry was taken as the time of incidence, and the origin and  destination of the journey were used along with the timetable to  estimate which services the journey was incident to. This processing  took some number of hours for the entire data set, but it was fully  automated. A large set of observations was obtained for which the  following data were measured or estimated: • Date and time of incidence, • Incidence headway, • SWT, • Location (i.e., station) of incidence (i.e., the journey’s origin), • Journey’s destination, and • Overground line to which the passenger was incident.

 



For journeys that required an interchange within the Overground  system (of which there are relatively few), the above data were   measured or estimated for only the first incidence event.

As a point of validation, Figure 2 plots distributions of incidence  headway for passengers on the Gospel Oak to Barking (GOB) and  North London Lines. The findings are consistent with expectations.  On the NLL, the mode of all the distributions is  $15~\mathrm{min}$ , reflect- ing the core service. The distribution is more concentrated during  the interpeak period, when there are no scheduled shuttles or spe- cials. The opposite is true on the GOB, which runs a regular 20-min   service in the peak periods but transitions to and from a 30-min  service in the interpeak period. The morning peak distribution is  somewhat more dispersed than that of the evening peak because the  morning peak includes a transition from 30-min headways in the  early morning.

### results

Consistent with the reviewed studies of passenger incidence behav- ior, the first results of interest are distributions of passenger inci- dence time over a given headway. Figure 3 plots these distributions  for the London Overground network by line and by time period.  In this plot, incidence times are normalized by the incidence head- way because different passengers experience different incidence  headways.

It is clear from Figure 3 that passenger incidence behavior, with  respect to published timetables, varies spatially and temporally across  the Overground network. Passenger incidence on the GOB and   Watford DC lines is much more peaked (i.e., timetable-dependent),  each with 20-min headways, than on the NLL, with 7- to 15-min  headways, during any time period. Also, the NLL is acknowledged by  Overground management to have the most serious reliability problems  (O. Bratton, personal communication, 2010). These variations are con- sistent with the literature that passenger incidence is more timetable- dependent with a more peaked distribution for longer headways and  for more reliable services.

Also consistent with the literature is that, for all lines, the distri- bution is more peaked in the morning peak period than in the eve- ning peak or midday interpeak periods. It appears that Overground  commuters in the morning peak period, more likely to have knowl- edge of the timetable and the service and more sensitive to time  saving, exhibit less random incidence behavior than do passengers  in other time periods, despite the shorter headways and less reliable  service found in the morning peak. The distribution in the late-night  period is also very peaked because of longer headways and possibly  higher sensitivity to time.

The peaks of all of the distributions are somewhat before the very  end of the headway, indicating some type of safety margin or wait- ing time–minimization behavior on the part of passengers. Many of  the distributions have small spikes at the beginning of the headway,  indicating possible late running awareness among some passengers.  While such awareness may in fact be found on the Overground, it is  also possible that it is the passengers themselves who are running late.  They may be planning to take a train scheduled to depart at a certain  time but, because of uncontrollable circumstances or their own poor  planning, arrive at the station shortly after that departure time.

Figure 4 shows the mean waiting time, for each line and time period,  under two models of passenger behavior and train operations: random  incidence versus timetable-dependent incidence. In the first case, it is  assumed that all passengers are randomly incident to constant head- way services, so the mean waiting time is calculated as half the mean  incidence headway. In the second case, no behavioral assumption is  made and all trains are assumed to run as per the timetable, so the  observed mean waiting time is calculated as the mean SWT. Figure 4  thus indicates the effects of the observed incidence behaviors  (compared with random incidence) on passenger waiting times.

On the NLL, the relatively slight skew in the incidence distribu- tions translates into relatively small impacts on waiting time. In the  morning peak, timetable dependence decreases waiting time by $7.2\%$   from 6.82 to $6.33\,\mathrm{min}$  (about 30 s). In the interpeak and evening peak  periods, the reductions are only  $0.2\%$  and $2.2\%$ , respectively. On the  GOB, however, the implications of timetable dependence are substan- tial. In the morning peak, the waiting time decreases by $29\%$  (3.1 min)  from 10.5 to  $7.4\,\mathrm{min}$ . In the interpeak and evening peak periods, the  reductions are  $20.4\%$  and  $17.5\%$ , respectively.

Transit assignment models often make an assumption that no ser- vice would be assigned a mean passenger waiting time over a certain  threshold, say,  $10\;\mathrm{min}$  in the London Overground case. The results  of this section lend support to this assumption. Regular weekday  headways on the Overground network go as high as  $30\,\mathrm{min}$ , but the  observed mean SWT is above 11 min in only two cases (on the West  London Line during the interpeak and evening time periods) and is  never above  $12.1\;\mathrm{min}$ .

### discussion and concLusions

This paper developed a methodology to relate disaggregate AFC  journey data to published timetables for the purpose of studying  passenger incidence behavior and applied this methodology to the  London Overground (16).

The following conclusions are drawn about the methodology  developed here. First, it can be used to study passenger incidence  behavior using large samples of disaggregate journey data from  AFC systems such as the Oyster smartcard system. Second, the  methodology can for each passenger journey estimate SWT and  incidence headway under heterogeneous conditions. Finally, this  methodology is able to efficiently process millions of data records  and can be implemented using open standard timetable formats and  free software tools.

As a limitation the proposed method hinges on the schedule- based assignment, for which there are unsolved methodological  issues such as station congestion or train crowding. The London  Overground is a relatively simple network (i.e., compared with the  London Underground), so that the methodology would not be sub- ject to those issues. As the inner workings of schedule-based assign- ment improve, the method of this paper will continue to become  more broadly applicable.

With respect to the London Overground, the following conclu- sions can be drawn. Generally, passenger incidence behavior varies  across the network and across times of day, and the differences reflect  different headways, reliability of service, and time of day (indicating  different knowledge of the network and sensitivity to time); these  findings reflect the findings in the literature to date. Specifically, inci- dence appears to be much less timetable dependent on the NLL than  on the other Overground lines because of shorter headways and less  reliable service on this line as compared with others. The method  and results of the paper also provide one service quality measure,  SWT, for the entire Overground network. On the GOB, Watford DC,  and West London lines, where incidence behavior is more timetable- dependent, passengers reduce their mean SWT by more than  $3\;\mathrm{min}$ ,  or up to  $30\%$ , during daytime hours compared with random inci- dence behavior. On the NLL, such reductions are much smaller, in  some cases nearly zero, in both relative and absolute terms.



The method developed in this paper can support further study  of passenger incidence behavior in a more systematic way; that is,  researchers no longer need to select a special subset of stations with  only one service pattern. The work of Bowman and Turnquist has  been influential in shaping the understanding of the relationships  between headway, reliability, passenger behavior, and waiting time  (1). Their work should be updated by using the method of this paper so  that large samples of passenger data across heterogeneous networks  can be easily analyzed. The London Overground network represents  an ideal opportunity to conduct such a study because its passengers  can clearly be studied via Oyster data, and its trains are tracked by a  computerized signaling system. After the East  London Line opened,  the network has headways ranging from 5 to  $30\ \mathrm{min}$  during most  hours of the day, thus providing a good range of variation.

Another application is to study the impact of real-time informa- tion. Many strides have been made toward informing passengers in  real time about the status of public transport services. Such infor- mation is now often distributed via in-station signs and announce- ments, as well as over the Internet to passengers’ computers and,  more importantly, mobile devices. It is crucial to advance the  understanding of passenger incidence to include the effects of real- time information. Such an advance in understanding will require  careful thinking and research designs, but it should be able to take  advantage of the methodology developed here. For example, actual  recorded train arrival and departure times (rather than published  timetables) could be used in the schedule-based assignment.

### acknowLedgMents

The authors thank Nigel Wilson for cosupervising the research proj- ect and providing critical input to the master’s thesis on which this  paper is based. The authors also thank Transport for London for pro- viding the financial and institutional support for this research, the  opportunity to work in the agency, and access to the data used in this  paper and specifically thank Oliver Bratton of London Overground  Rail Operations Ltd for insight into the issues and the dynamics of  the London Overground network.



### references

 1. Bowman, L. A., and M. A. Turnquist. Service Frequency, Schedule  Reliability, and Passenger Wait Times at Transit Stops. Transportation  Research Part A, Vol. 15, No. 6, 1981, pp. 465–471.

 2. Furth, P. G., and T. H. J. Muller. Service Reliability and Hidden Wait- ing Time. In Transportation Research Record 1955, TRB, National  Research Council, Washington, D.C., 2006, pp. 79–87.

 3. Larson, R. C., and A. R. Odoni. Urban Operations Research, 2nd ed.  Dynamic Ideas, Belmont, Mass., 2007.

 4. Osuna, E. E., and G. F. Newell. Control Strategies for an Idealized Pub- lic Transportation System. Transportation Science, Vol. 6, No. 1, 1972,  pp. 52–71.

 5. Jolliffe, J. K., and T. P. Hutchinson. A Behavioral Explanation of the  Association Between Bus and Passenger Arrivals at a Bus Stop. Trans- portation Science, No. 9, 1975, pp. 248–282.

 6. Luethi, M., U. A. Weidmann, and A. Nash. Passenger Arrival Rates at  Public Transport Stations. Presented at 86th Annual Meeting of the  Transportation Research Board, Washington, D.C., 2007.

 7. Csikos, D. R., and G. Currie. Investigating Consistency in Transit Pas- senger Arrivals: Insights from Longitudinal Automated Fare Collection  Data. In Transportation Research Record: Journal of the Transporta- tion Research Board, No. 2042, Transportation Research Board of the  National Academies, Washington, D.C., 2008, pp. 12–19.

 8. Csikos, D. R., and G. Currie. The Impacts of Transit Reliability on Wait  Time: Insights from Automated Fare Collection System Data. Pre- sented at 86th Annual Meeting of the Transportation Research Board,   Washington, D.C., 2007.

 9. Spiess, H., and M. Florian. Optimal Strategies: A New Assignment  Model for Transit Networks. Transportation Research Part B, Vol. 23,  No. 2, 1989, pp. 83–102.

10. Nguyen, S., and S. Pallottino. Equilibrium Traffic Assignment for  Large-Scale Transit Networks. European Journal of Operations Research,  Vol. 37, 1988, pp. 176–186.

11. Nuzzolo, A., and U. Crisalli. The Schedule-Based Approach in Dynamic  Transit Modeling: A General Overview. In Schedule-Based Dynamic  Transit Modeling: Theory and Applications (N. H. M. Wilson and  A. Nuzzolo, eds.). Operations Research/Computer Science Interface  Series, Kluwer Academic Publishers, 2004, pp. 1–24.

12. Bertsimas, D., and J. N. Tsitsiklis. Introduction to Linear Optimization.  Dynamic Ideas, Belmont, Mass., 1997.

13. Graphserver. Graphserver: The Open-Source Multi-Modal Trip Planner.  http://bmander.github.com/graphserver/. Accessed 2009.

14. Google. General Transit Feed Specification Reference. http://code. google.com/transit/spec/transit_feed_specification.html. Accessed  Nov. 13, 2009.

15. Perl.org. The Perl Programming Language. http://www.perl.org/.  Accessed 2010.

16. Frumin, M. Automatic Data for Applied Railway Management: Passen- ger Demand, Service Quality Measurement, and Tactical Planning on  the London Overground Network. MS thesis. Massachusetts Institute of  Technology, Cambridge, Mass., 2010.

The Transit Management and Performance Committee peer-reviewed this paper.
# CASE REPORT

# Effect of rituximab on clinical and laboratory features of antiphospholipid syndrome: a case report and a review of literature

GL Erre, S Pardini2, R Faedda3 and G Passiu1

1Cattedra e Scuola di Specializzazione di Reumatologia; 2Istituto di Ematologia; and 3Istituto di Patologia Medica e Metodologia Clinica  Università degli Studi di Sassari, Italy

Antiphospholipid syndrome (APS) is an autoimmune disorder characterized by a hypercoagulable state related to persistently elevated levels of antiphospholipid antibodies (aPL). Current treatment for APS is only partially effective and new therapies are strongly needed.

We report on a case of a 50 years old man with APS who suffered from recurrent thromboembolic episodes despite conventional anticoagulant treatment. Eight years after the ﬁrst thrombotic manifestation he was diagnosed with a large B cell non-Hodgkin lymphoma. Treatment with CHOP (cyclophosphamide, doxorubicin, vincristine and prednisone) plus rituximab was started with partial clinical remission of lymphoma and normalization of aPL levels with a three years follow-up period free of thrombotic episodes.

A review of the literature revealed that only 12 case reports on the use of rituximab in patients with primary, secondary and catastrophic APS have been published. Current knowledge clearly suggests the need for clinical trials to evaluate the effect of rituximab in the treatment of resistant APS.Lupus (2008) 17, 50–55.

Key words: anticardiolipin (antibodies); antiphospholipid antibodies; rituximab; lupus anticoagulant; lymphoma

## Introduction

The antiphospholipid syndrome (APS) is an  autoimmune disorder characterized by arterial and venous thrombosis, recurrent fetal loss and persistently positive serum antiphospholipid antibodies (aPL). A life-threatening form of APS, named catastrophic APS (CAPS) is due to a widespread thrombotic process and may present with multiple organ failure.

APS may develop independently of any underlying disease (primary APS or PAPS), in association with autoimmune diseases, mainly systemic lupus erythe- matosus (SLE) or malignancies (secondary APS or SAPS).

Some authors have reported the efﬁcacy of chemotherapy including rituximab, a chimeric mono- clonal antibody directed against CD20 widely used in

\*Correspondence:Gian Luca Erre,MD,Cattedra e Scuola di Specializzazione di Reumatologia, Università degli Studi di Sassari V.le San Pietro 8, 07100 Sassari, Italy.

E-mail: e.gianluca@libero.it Received 26 June 2007; accepted 19 September 2007

the treatment of B cell malignancies, on clinical manifestations of APS or on reducing the levels of aPL. Prompted by this, several authors have also demonstrated the efﬁcacy of rituximab on the treat- ment of several autoimmune conditions, including rheumatoid arthritis, SLE, Sjögren syndrome and vasculitis. To date similar evidences in the treatment of APS are limited and mainly anecdotal.

## Methods

Lupus anticoagulant was determined by the Kaolin- clotting time (KCT) and a dilute Russel viper venom time (dRVVT). The anticardiolipin (aCL) IgG and  $\mathrm{Ig}\mathbf{M}$  antibodies and the anti- $\beta_{2}$ -Glycoprotein I antibodies were determined by solid-phase enzyme-linked immunosorbent assays (GA, generic assays GmBH, Germany).

Each episode of deep venous thrombosis (DVT) was proven with venous compression ultrasonography or angiography. Noncompressibility of a previously normalized venous segment and enlargement of thrombus thickness ( $(>\,\mathrm{or}=2\,\mathrm{mm})$  were considered diagnostic of proximal DVT recurrence. Pulmonary embolisms were conﬁrmed by scintigram/ventilation- perfusion scan and/or thorax CT scan.



## Case report

The patient, a 50-year-old man, was previously known with a diagnosis of primary APS. In 1995, he was urgently admitted to our hospital for a left common iliac DVT and acute pulmonary embolism (PE). Anticardiolipin antibodies (aCL) were elevated and lupus anticoagulant (LAC) was demonstrated in the serum of the patient on two different occasions. Thus a diagnosis of PAPS according to the international con- sensus criteria (Sapporo)3 was made.

Lifelong anticoagulant therapy with a moderate intensity warfarin (target international normalized ratio (INR) of 2.0–3.0) was started but, despite treat- ment, three further episodes of DVT occurred. In 2002 a new episode of PE supervened and an inferior vena cava ﬁlter was placed.

In 2003, he was admitted to our institute for fatigue, fever and upper abdominal pain radiating to the inter- scapular region. Physical examination revealed severe splenomegaly and axillary lymphoadenopathy.

Laboratory investigations conﬁrmed positivity for aPL (aCL, LAC and anti- $\upbeta_{2}G P I$  antibodies) and showed positive antinuclear antibodies (ANA 1:160) and anti-double stranded DNA antibodies.

An abdominal Doppler ultrasonography revealed a considerable enlargement of the spleen and multiple focal lesions while angiography showed thrombosis of the splenic vein. Working diagnosis included lym- phoma and multiple splenic infarcts. Splenectomy was not performed because the risks for perioperative thromboembolic and hemorrhagic complications were felt to be very high.

An axillary lymph node excision showed histology consistent with large B cell non-Hodgkin lymphoma while staging bone marrow biopsy was negative. Chest X-ray demonstrated multiple pulmonary nodules prompting an extensive work-up with whole body CT scan and gallium scintigraphy for stadiation of lym- phoma that conﬁrmed pulmonary localization of lym- phoproliferative disease (stage IV). Because of persistent headaches the patient underwent brain MRI and partial thrombosis of the left transverse cerebral sinus was demonstrated. On the basis of clinical, instrumental and laboratory evidences a diagnosis of  B cell non-Hodgkin lymphoma and primary APS was made in this patient.

After obtaining informed consent, chemotherapy in- cluding CHOP (cyclophosphamide (Cyc), doxorubicin, vincristine and prednisone) plus rituximab was started. Rituximab was scheduled at a dose of  $375\,\mathrm{mg}/\mathrm{m}^{2}$ every three weeks for a total of six doses. Rituximab was well tolerated with the exception of a transitory hypotension during the ﬁrst infusion.



After four infusions the patient developed a com- plete thrombosis of the left femoral vein and a partial thrombosis of the left iliac vein. After the sixth dose of rituximab aPL levels dramatically fell (Figure 1 and Table 1) and no further thromboembolic episodes occurred during follow-up.

The administration of rituximab allowed achieving a partial clinical remission of lymphoma. Unfortunately in 2006 the patient experienced a  recurrence of pulmonary lymphoma detected by  surveillance CT and positron emission tomography (PET). He ﬁnally died from acute respiratory failure due to extensive lymphomatous pulmonary involve- ment. An autopsy was not carried out.

## Literature review and discussion

We searched the Medline database (from 1995, year of introduction of rituximab MeSH term, to April 2007) to determine the effect of rituximab on aPL levels, thromboembolic events and thrombocytopenia in patient affected by primary APS, secondary APS and catastrophic APS. We use the following MeSH terms alone or in combination: rituximab, APS, aPL, LAC, anticardiolipin antibody, DVT, pulmonary embolism, stroke and SLE. We further reviewed reference lists. The search was limited to article in English.

We found 12 case reports published so far (Table 2). No controlled trials have been published, limiting 

NA: not available.

current evidence to anecdotal cases, an obvious cause of publication bias due to the exclusion of cases in which treatment with rituximab failed to produce clin- ical improvement.

Thrombocytopenia is a common laboratory mani- festation of APS and in patients affected with SLE is frequently related to the presence of aPL. In patients with severe thrombocytopenia and high risk for bleed- ing,various immunosuppressive treatments are demonstrated to be effective in improving platelet count. However, a not negligible percentage of patients become resistant to immunosuppression and splenec- tomy cannot be performed because of high risk of bleeding and thrombotic episodes. In these patients, a rational for the use of rituximab can be based on its effect on reducing aPL levels which are in turn  postulated to be responsible for platelet destruction by binding platelet membrane.

Trappe and co-workers4 demonstrated a close corre- lation between aCL title reduction and platelets count improvement in a patient with primary APS treated with rituximab for thrombocytopenia refractory to conventional therapies and similar encouraging results were obtained by other authors.

Ames et al. reported only a partial effect of ritux- imab on platelets count consistent with evidence coming from larger trials that demonstrated inefﬁcacy of rituximab in about a half of adult patients with idio- pathic thrombocytopenic purpura (ITP).

Patients with primary or secondary APS can develop during the course of disease isolated autoim- mune haemolytic anaemia (AHA) or an association of AHA and ITP.

Haemolysis may be due to a premature erythro- cyte sequestration and destruction by a pathogenic  mechanism similar to that suggested for immune thrombocytopenia.

Erdozain et al. demonstrated resolution of refrac- tory AHA in a patient with primary APS following treatment with standard dose of rituximab and, it is interesting to note the lowering effect of this  drug on the levels of IgM aCL that decreased from  600 MPL to 100 MPL consistent with data reported  by Tomietto et al. in patient with SLE and secondary APS.



This temporal association is very intriguing in sup- porting a role of antibody reduction on improvement of clinical related manifestations, although the hypoth- esis that the two phenomenons are merely associated cannot be surely refuted.

On the other hand some authors failed to demon- strate a persistent reduction of  $\mathrm{aPL}^{4,5,11}$  this might be due to the fact that rituximab does not affect long-lived plasma cells12 acting mainly on B cell leaving lymph node germinal centre.

Arterial and venous thromboses are the clinical hallmarks of APS and guide therapeutic choice. Gold standard for the treatment of venous thrombosis and stroke is an indeﬁnite anticoagulation but in spite of a seemingly adequate therapy in some patient vessel thrombosis and associated embolism tend to recur. Different immunosuppressants, plasma exchange or i.v. immunoglobulins have been used as rescue therapy particularly in patients with life-threatening throm- botic events and in the catastrophic APS.

Prompted by a lack of efﬁcacy with previous stan- dard and conventional treatment several authors observed a dramatic resolution of recurrent and/or severe thrombotic episodes in patients with APS following rituximab introduction. A variable follow-up free of thromboembolic manifestations ranging from 10 to 36 months has been reported.

This ﬁnding, if validated from large clinical trials, could have favourable implications mainly for the treatment of catastrophic APS as reported by Rubenstein et al.

Concomitant immunosuppressors such as cyclo- sporine  $\mathrm{A}^{5}$  or plasmapheresis treatment, might  inﬂuence overall effect of rituximab.

In this regard our protocol included Cyc, so one could think that effect on aPL and clinical features of APS may partially be attributed to chemotherapy. Previous authors reported inefﬁcacy of conventional CHOP regimen on improvement of APS-related hypercoagulable state, therefore suggesting a promi- nent role of anti-CD20 on the induction of this effect.

On the other hand Leandro et al. reported a major clinical improvement in patients affected by rheuma- toid arthritis when Cyc was added to rituximab.

Most authors used rituximab weekly for four weeks and in some cases a further infusion at three monthly intervals. We administered rituximab at three weekly intervals and for a total of six infusions.

Interestingly, in this regard, we did not obtain clinical and serological remission (a further episode of DVT occurred and only a partial reduction of aPL titers was obtained) of APS after the ﬁrst four doses possibly due to the fact that we did not reach a complete B cell depletion. Unfortunately this hypothesis cannot be supported by B cell plasmatic levels because of the retrospective nature of this case report.

Prospective studies are needed to evaluate efﬁcacy and safety of rituximab alone or in combination with other immunosuppressants and address the issues related to the optimal regimen of persistent B cell depletion achieved by maintenance therapy in these patients.

## Conclusions

The case we described and our analysis of the literature are encouraging with regard to the efﬁcacy of rituximab on controlling clinical features of APS and lowering aPL level. Current evidence is only limited to anecdotal cases and randomized controlled trials in patients with APS resistant to the standard therapeutic approach are strongly needed.

## Acknowledgements

The authors thank Giovanni Sanna, MD, for helpful discussion and critical review of this manuscript.

## References

1 Hughes G. The antiphospholipid syndrome: Ten years on. Lancet 1993; 342: 341–344.

2 Veneri D, Ambrosetti A, Franchini M, Mosna F, Poli G, Pizzolo G. Remission of severe antiphospholipid syndrome associated with non- Hodgkin B cell lymphoma after combined treatment with rituximab and chemotherapy. Haematologica 2005; 90: 16–17.

3 Wilson WA, Gharavi AE, Koike T et al. International consensus state- ment on preliminary classiﬁcation criteria for deﬁnite antiphospholipid syndrome: report of an international workshop. Arthrits Rheum 1999; 42: 1309–1311.

4 Trappe R, Loew A, Thuss-Patience P, Dörken B, Riess H. Successful treatment of thrombocytopenia in primary antiphospholipid antibody syndrome with the anti-CD20 antibody rituximab-monitoring of antiphospholipid and anti-GP antibodies: a case report. Ann Hematol 2006; 85: 134–135.

5 Tomietto P, Gremese E, Tolusso B, Venturini P, De Vita S, Ferraccioli G. B cell depletion may lead to normalization of anti- platelet, anti-erythrocyte and antiphospholipid antibodies in systemic lupus erythematosus. Thromb Haemost 2004; 92: 1150–1153.

6 Rubenstein E, Arkfeld D, Metyas S, Shinada S, Ehresmann S, Liebman HA. Rituximab treatment for resistant antiphospholipid syndrome.  J Rheumatol 2006; 33: 355–357.

7 Ames PRJ, Tommasino C, Fossati G, Scenna G, Brancaccio V, Ferrara F. Limited effect of rituximab on thrombocytopenia and anticardiolipin antibodies in a patient with primary antiphospholipid syndrome. Ann Hematol 2007; 86: 227–228.

8 Cooper N, Stasi R, Cunningham-Rundles S et al. The efﬁcacy and safety of B cell depletion with anti-CD20 monoclonal antibody in adults with chronic immune thrombocytopenic purpura. Br J Hematol 2004; 125: 232–239.

9 Braendstrup P, Bjerrum OW, Nielsen OJ et al. Rituximab chimeric anti- CD20 monoclonal antibody treatment for adult refractory idiopathic thrombocytopenic purpura. Am J Hematol 2005; 78: 275–280.

10 Erdozain JG, Ruiz-Irastorza G, Egurbide MV, Aguirre C. Sustained response to rituximab of autoimmune haemolytic anaemia associated with antiphospholipid syndrome. Haematologica 2004; 89: ECR34.

11 Ahn ER, Lander G, Bidot C, Wenche Jy, Ahn YS. Long-term remission from life-threatening hypercoagulable state associated with lupus anti- coagulant (LA) following rituximab therapy. Am J Hematol 2005; 78: 127–129.

12 Manz RA, Arce S, Cassese G, Hauser AE, Hiepe F, Radbruch A. Humoral immunity and long-lived plasma cells. Curr Opinion Immunol 2002; 14: 517–521.

13 Lim W, Crowther MA, Eikelboom JW. Management of antiphospholipid syndrome. JAMA 2006; 295: 1050–1057.

14 Harner KC, Jackson LW, Drabick JJ. Normalization of anticardiolipin antibodies following rituximab therapy for marginal zone lymphoma in a patient with Sjögren’s syndrome. Rheumatology 2004; 43: 1309–1310. 15 Weide R, Heymanns J, Pandorf A, Koppler H. Successful long-term treatment of systemic lupus erythematosus with rituximab maintenance therapy. Lupus 2003; 12: 779–782.

16 Leandro MJ, Edwards JC, Cambridge G. Clinical outcome in 22 patients with rheumatoid arthritis treated with B lymphocyte depletion. Ann Rheum Dis 2002; 61: 863–866.

17 Anandacoomarasamy A, Gibson J, Mc Gill N. “Cure” of life-threatening antiphospholipid syndrome with rituximab. Intern Med J 2006; 36: 474–475.
# CASE REPORT

# Effect of rituximab on clinical and laboratory features of antiphospholipid syndrome: a case report and a review of literature

GL Erre, S Pardini2, R Faedda3 and G Passiu1

1Cattedra e Scuola di Specializzazione di Reumatologia; 2Istituto di Ematologia; and 3Istituto di Patologia Medica e Metodologia Clinica  Università degli Studi di Sassari, Italy

Antiphospholipid syndrome (APS) is an autoimmune disorder characterized by a hypercoagulable state related to persistently elevated levels of antiphospholipid antibodies (aPL). Current treatment for APS is only partially effective and new therapies are strongly needed.

We report on a case of a 50 years old man with APS who suffered from recurrent thromboembolic episodes despite conventional anticoagulant treatment. Eight years after the ﬁrst thrombotic manifestation he was diagnosed with a large B cell non-Hodgkin lymphoma. Treatment with CHOP (cyclophosphamide, doxorubicin, vincristine and prednisone) plus rituximab was started with partial clinical remission of lymphoma and normalization of aPL levels with a three years follow-up period free of thrombotic episodes.

A review of the literature revealed that only 12 case reports on the use of rituximab in patients with primary, secondary and catastrophic APS have been published. Current knowledge clearly suggests the need for clinical trials to evaluate the effect of rituximab in the treatment of resistant APS.Lupus (2008) 17, 50–55.

Key words: anticardiolipin (antibodies); antiphospholipid antibodies; rituximab; lupus anticoagulant; lymphoma

## Introduction

The antiphospholipid syndrome (APS) is an  autoimmune disorder characterized by arterial and venous thrombosis, recurrent fetal loss and persistently positive serum antiphospholipid antibodies (aPL). A life-threatening form of APS, named catastrophic APS (CAPS) is due to a widespread thrombotic process and may present with multiple organ failure.

APS may develop independently of any underlying disease (primary APS or PAPS), in association with autoimmune diseases, mainly systemic lupus erythe- matosus (SLE) or malignancies (secondary APS or SAPS).

Some authors have reported the efﬁcacy of chemotherapy including rituximab, a chimeric mono- clonal antibody directed against CD20 widely used in

\*Correspondence:Gian Luca Erre,MD,Cattedra e Scuola di Specializzazione di Reumatologia, Università degli Studi di Sassari V.le San Pietro 8, 07100 Sassari, Italy.

E-mail: e.gianluca@libero.it Received 26 June 2007; accepted 19 September 2007

the treatment of B cell malignancies, on clinical manifestations of APS or on reducing the levels of aPL. Prompted by this, several authors have also demonstrated the efﬁcacy of rituximab on the treat- ment of several autoimmune conditions, including rheumatoid arthritis, SLE, Sjögren syndrome and vasculitis. To date similar evidences in the treatment of APS are limited and mainly anecdotal.

## Methods

Lupus anticoagulant was determined by the Kaolin- clotting time (KCT) and a dilute Russel viper venom time (dRVVT). The anticardiolipin (aCL) IgG and  $\mathrm{Ig}\mathbf{M}$  antibodies and the anti- $\beta_{2}$ -Glycoprotein I antibodies were determined by solid-phase enzyme-linked immunosorbent assays (GA, generic assays GmBH, Germany).

Each episode of deep venous thrombosis (DVT) was proven with venous compression ultrasonography or angiography. Noncompressibility of a previously normalized venous segment and enlargement of thrombus thickness ( $(>\,\mathrm{or}=2\,\mathrm{mm})$  were considered diagnostic of proximal DVT recurrence. Pulmonary embolisms were conﬁrmed by scintigram/ventilation- perfusion scan and/or thorax CT scan.



## Case report

The patient, a 50-year-old man, was previously known with a diagnosis of primary APS. In 1995, he was urgently admitted to our hospital for a left common iliac DVT and acute pulmonary embolism (PE). Anticardiolipin antibodies (aCL) were elevated and lupus anticoagulant (LAC) was demonstrated in the serum of the patient on two different occasions. Thus a diagnosis of PAPS according to the international con- sensus criteria (Sapporo)3 was made.

Lifelong anticoagulant therapy with a moderate intensity warfarin (target international normalized ratio (INR) of 2.0–3.0) was started but, despite treat- ment, three further episodes of DVT occurred. In 2002 a new episode of PE supervened and an inferior vena cava ﬁlter was placed.

In 2003, he was admitted to our institute for fatigue, fever and upper abdominal pain radiating to the inter- scapular region. Physical examination revealed severe splenomegaly and axillary lymphoadenopathy.

Laboratory investigations conﬁrmed positivity for aPL (aCL, LAC and anti- $\upbeta_{2}G P I$  antibodies) and showed positive antinuclear antibodies (ANA 1:160) and anti-double stranded DNA antibodies.

An abdominal Doppler ultrasonography revealed a considerable enlargement of the spleen and multiple focal lesions while angiography showed thrombosis of the splenic vein. Working diagnosis included lym- phoma and multiple splenic infarcts. Splenectomy was not performed because the risks for perioperative thromboembolic and hemorrhagic complications were felt to be very high.

An axillary lymph node excision showed histology consistent with large B cell non-Hodgkin lymphoma while staging bone marrow biopsy was negative. Chest X-ray demonstrated multiple pulmonary nodules prompting an extensive work-up with whole body CT scan and gallium scintigraphy for stadiation of lym- phoma that conﬁrmed pulmonary localization of lym- phoproliferative disease (stage IV). Because of persistent headaches the patient underwent brain MRI and partial thrombosis of the left transverse cerebral sinus was demonstrated. On the basis of clinical, instrumental and laboratory evidences a diagnosis of  B cell non-Hodgkin lymphoma and primary APS was made in this patient.

After obtaining informed consent, chemotherapy in- cluding CHOP (cyclophosphamide (Cyc), doxorubicin, vincristine and prednisone) plus rituximab was started. Rituximab was scheduled at a dose of  $375\,\mathrm{mg}/\mathrm{m}^{2}$ every three weeks for a total of six doses. Rituximab was well tolerated with the exception of a transitory hypotension during the ﬁrst infusion.



After four infusions the patient developed a com- plete thrombosis of the left femoral vein and a partial thrombosis of the left iliac vein. After the sixth dose of rituximab aPL levels dramatically fell (Figure 1 and Table 1) and no further thromboembolic episodes occurred during follow-up.

The administration of rituximab allowed achieving a partial clinical remission of lymphoma. Unfortunately in 2006 the patient experienced a  recurrence of pulmonary lymphoma detected by  surveillance CT and positron emission tomography (PET). He ﬁnally died from acute respiratory failure due to extensive lymphomatous pulmonary involve- ment. An autopsy was not carried out.

## Literature review and discussion

We searched the Medline database (from 1995, year of introduction of rituximab MeSH term, to April 2007) to determine the effect of rituximab on aPL levels, thromboembolic events and thrombocytopenia in patient affected by primary APS, secondary APS and catastrophic APS. We use the following MeSH terms alone or in combination: rituximab, APS, aPL, LAC, anticardiolipin antibody, DVT, pulmonary embolism, stroke and SLE. We further reviewed reference lists. The search was limited to article in English.

We found 12 case reports published so far (Table 2). No controlled trials have been published, limiting 

NA: not available.

current evidence to anecdotal cases, an obvious cause of publication bias due to the exclusion of cases in which treatment with rituximab failed to produce clin- ical improvement.

Thrombocytopenia is a common laboratory mani- festation of APS and in patients affected with SLE is frequently related to the presence of aPL. In patients with severe thrombocytopenia and high risk for bleed- ing,various immunosuppressive treatments are demonstrated to be effective in improving platelet count. However, a not negligible percentage of patients become resistant to immunosuppression and splenec- tomy cannot be performed because of high risk of bleeding and thrombotic episodes. In these patients, a rational for the use of rituximab can be based on its effect on reducing aPL levels which are in turn  postulated to be responsible for platelet destruction by binding platelet membrane.

Trappe and co-workers4 demonstrated a close corre- lation between aCL title reduction and platelets count improvement in a patient with primary APS treated with rituximab for thrombocytopenia refractory to conventional therapies and similar encouraging results were obtained by other authors.

Ames et al. reported only a partial effect of ritux- imab on platelets count consistent with evidence coming from larger trials that demonstrated inefﬁcacy of rituximab in about a half of adult patients with idio- pathic thrombocytopenic purpura (ITP).

Patients with primary or secondary APS can develop during the course of disease isolated autoim- mune haemolytic anaemia (AHA) or an association of AHA and ITP.

Haemolysis may be due to a premature erythro- cyte sequestration and destruction by a pathogenic  mechanism similar to that suggested for immune thrombocytopenia.

Erdozain et al. demonstrated resolution of refrac- tory AHA in a patient with primary APS following treatment with standard dose of rituximab and, it is interesting to note the lowering effect of this  drug on the levels of IgM aCL that decreased from  600 MPL to 100 MPL consistent with data reported  by Tomietto et al. in patient with SLE and secondary APS.



This temporal association is very intriguing in sup- porting a role of antibody reduction on improvement of clinical related manifestations, although the hypoth- esis that the two phenomenons are merely associated cannot be surely refuted.

On the other hand some authors failed to demon- strate a persistent reduction of  $\mathrm{aPL}^{4,5,11}$  this might be due to the fact that rituximab does not affect long-lived plasma cells12 acting mainly on B cell leaving lymph node germinal centre.

Arterial and venous thromboses are the clinical hallmarks of APS and guide therapeutic choice. Gold standard for the treatment of venous thrombosis and stroke is an indeﬁnite anticoagulation but in spite of a seemingly adequate therapy in some patient vessel thrombosis and associated embolism tend to recur. Different immunosuppressants, plasma exchange or i.v. immunoglobulins have been used as rescue therapy particularly in patients with life-threatening throm- botic events and in the catastrophic APS.

Prompted by a lack of efﬁcacy with previous stan- dard and conventional treatment several authors observed a dramatic resolution of recurrent and/or severe thrombotic episodes in patients with APS following rituximab introduction. A variable follow-up free of thromboembolic manifestations ranging from 10 to 36 months has been reported.

This ﬁnding, if validated from large clinical trials, could have favourable implications mainly for the treatment of catastrophic APS as reported by Rubenstein et al.

Concomitant immunosuppressors such as cyclo- sporine  $\mathrm{A}^{5}$  or plasmapheresis treatment, might  inﬂuence overall effect of rituximab.

In this regard our protocol included Cyc, so one could think that effect on aPL and clinical features of APS may partially be attributed to chemotherapy. Previous authors reported inefﬁcacy of conventional CHOP regimen on improvement of APS-related hypercoagulable state, therefore suggesting a promi- nent role of anti-CD20 on the induction of this effect.

On the other hand Leandro et al. reported a major clinical improvement in patients affected by rheuma- toid arthritis when Cyc was added to rituximab.

Most authors used rituximab weekly for four weeks and in some cases a further infusion at three monthly intervals. We administered rituximab at three weekly intervals and for a total of six infusions.

Interestingly, in this regard, we did not obtain clinical and serological remission (a further episode of DVT occurred and only a partial reduction of aPL titers was obtained) of APS after the ﬁrst four doses possibly due to the fact that we did not reach a complete B cell depletion. Unfortunately this hypothesis cannot be supported by B cell plasmatic levels because of the retrospective nature of this case report.

Prospective studies are needed to evaluate efﬁcacy and safety of rituximab alone or in combination with other immunosuppressants and address the issues related to the optimal regimen of persistent B cell depletion achieved by maintenance therapy in these patients.

## Conclusions

The case we described and our analysis of the literature are encouraging with regard to the efﬁcacy of rituximab on controlling clinical features of APS and lowering aPL level. Current evidence is only limited to anecdotal cases and randomized controlled trials in patients with APS resistant to the standard therapeutic approach are strongly needed.

## Acknowledgements

The authors thank Giovanni Sanna, MD, for helpful discussion and critical review of this manuscript.

## References

1 Hughes G. The antiphospholipid syndrome: Ten years on. Lancet 1993; 342: 341–344.

2 Veneri D, Ambrosetti A, Franchini M, Mosna F, Poli G, Pizzolo G. Remission of severe antiphospholipid syndrome associated with non- Hodgkin B cell lymphoma after combined treatment with rituximab and chemotherapy. Haematologica 2005; 90: 16–17.

3 Wilson WA, Gharavi AE, Koike T et al. International consensus state- ment on preliminary classiﬁcation criteria for deﬁnite antiphospholipid syndrome: report of an international workshop. Arthrits Rheum 1999; 42: 1309–1311.

4 Trappe R, Loew A, Thuss-Patience P, Dörken B, Riess H. Successful treatment of thrombocytopenia in primary antiphospholipid antibody syndrome with the anti-CD20 antibody rituximab-monitoring of antiphospholipid and anti-GP antibodies: a case report. Ann Hematol 2006; 85: 134–135.

5 Tomietto P, Gremese E, Tolusso B, Venturini P, De Vita S, Ferraccioli G. B cell depletion may lead to normalization of anti- platelet, anti-erythrocyte and antiphospholipid antibodies in systemic lupus erythematosus. Thromb Haemost 2004; 92: 1150–1153.

6 Rubenstein E, Arkfeld D, Metyas S, Shinada S, Ehresmann S, Liebman HA. Rituximab treatment for resistant antiphospholipid syndrome.  J Rheumatol 2006; 33: 355–357.

7 Ames PRJ, Tommasino C, Fossati G, Scenna G, Brancaccio V, Ferrara F. Limited effect of rituximab on thrombocytopenia and anticardiolipin antibodies in a patient with primary antiphospholipid syndrome. Ann Hematol 2007; 86: 227–228.

8 Cooper N, Stasi R, Cunningham-Rundles S et al. The efﬁcacy and safety of B cell depletion with anti-CD20 monoclonal antibody in adults with chronic immune thrombocytopenic purpura. Br J Hematol 2004; 125: 232–239.

9 Braendstrup P, Bjerrum OW, Nielsen OJ et al. Rituximab chimeric anti- CD20 monoclonal antibody treatment for adult refractory idiopathic thrombocytopenic purpura. Am J Hematol 2005; 78: 275–280.

10 Erdozain JG, Ruiz-Irastorza G, Egurbide MV, Aguirre C. Sustained response to rituximab of autoimmune haemolytic anaemia associated with antiphospholipid syndrome. Haematologica 2004; 89: ECR34.

11 Ahn ER, Lander G, Bidot C, Wenche Jy, Ahn YS. Long-term remission from life-threatening hypercoagulable state associated with lupus anti- coagulant (LA) following rituximab therapy. Am J Hematol 2005; 78: 127–129.

12 Manz RA, Arce S, Cassese G, Hauser AE, Hiepe F, Radbruch A. Humoral immunity and long-lived plasma cells. Curr Opinion Immunol 2002; 14: 517–521.

13 Lim W, Crowther MA, Eikelboom JW. Management of antiphospholipid syndrome. JAMA 2006; 295: 1050–1057.

14 Harner KC, Jackson LW, Drabick JJ. Normalization of anticardiolipin antibodies following rituximab therapy for marginal zone lymphoma in a patient with Sjögren’s syndrome. Rheumatology 2004; 43: 1309–1310. 15 Weide R, Heymanns J, Pandorf A, Koppler H. Successful long-term treatment of systemic lupus erythematosus with rituximab maintenance therapy. Lupus 2003; 12: 779–782.

16 Leandro MJ, Edwards JC, Cambridge G. Clinical outcome in 22 patients with rheumatoid arthritis treated with B lymphocyte depletion. Ann Rheum Dis 2002; 61: 863–866.

17 Anandacoomarasamy A, Gibson J, Mc Gill N. “Cure” of life-threatening antiphospholipid syndrome with rituximab. Intern Med J 2006; 36: 474–475.
Acta Crystallographica Section F Structural Biology and Crystallization Communications

ISSN 1744-3091

Yohei Mukai, Teruya Nakamura, Yasuo Yoshioka, Shin-ichi Tsunoda, Haruhiko Kamada, Shinsaku Nakagawa, Yuriko Yamagata and Yasuo Tsutsumi\*

 



Graduate School of Pharmaceutical Sciences, Osaka University, Japan, bLaboratory of Pharmaceutical Proteomics, National Institute of Biomedical Innovation (NiBio), Japan, Graduate School of Pharmaceutical Sciences, Kumamoto University, Japan, and dThe Center of Advanced Medical Engineering and Informatics, Osaka University, Japan Correspondence e-mail: ytsutsumi@phs.osaka-u.ac.jp Received 29 September 2008 Accepted 7 February 2009

 



# Crystallization and preliminary X-ray analysis of the tumour necrosis factor–tumour necrosis factor receptor type 2 complex

Tumour necrosis factor receptor type 2 (TNFR2, TNFRSF1B) is an essential receptor for various host-defence functions of tumour necrosis factor  $\alpha$  (TNF). As part of studies to determine the structure of TNFR2, the formation, crystallization and preliminary  $X.$ -ray diffraction analysis of the TNF–TNFR2 complex are described. The TNF–TNFR2 complex, which comprises one TNF trimer and three TNFR2 monomers, was conﬁrmed and puriﬁed by size- exclusion chromatography. Crystals of the TNF–TNFR2 complex were obtained using polyethylene glycol 3350 as a precipitant. The crystal belonged to space group  $P2_{1}2_{1}2_{1}$ , with unit-cell parameters  $a\ =\ 74.5$ ,  $b\ =\ 117.4$ ,  $\mathrm{~\bar{~}{~c~}~}=\,246.8\:\mathrm{~\overset~{A}~{~}~}$ . Assuming the presence of two TNF–TNFR2 complexes in the asymmetric unit, the Matthews coefﬁcient  $V_{\mathbf{M}}$  was  $2.49\;\mathring{\mathrm{A}}^{3}\;\mathrm{Da}^{-1}$  and the solvent content of the crystal was  $50.7\%$ . The crystal diffracted to  $2.95\mathrm{~\AA~}$ resolution.

## 1. Introduction

Tumour necrosis factor  $\alpha$  (TNF) is a major inﬂammatory cytokine that plays a central role in host defence and inﬂammation (Aggarwal, 2003). Overexpression of TNF is closely associated with inﬂammatory diseases such as rheumatoid arthritis and inﬂammatory bowel disease (Feldmann & Maini, 2003). Therefore, anti-TNF antibodies and soluble TNF receptors (TNFR) that interfere with the activity of TNF have been used to treat various inﬂammatory diseases (Feldmann, 2002). However, anti-TNF therapy inhibits both types of TNFR (TNFR1 and TNFR2), often causing serious side effects.

Previous studies using animal models of diseases such as arthritis and hepatitis have demonstrated that TNFR1 plays a predominant role in the pathogenesis and exacerbation of inﬂammation (Mori et al., 1996; Leist et al., 1995). In contrast, TNFR2 is crucial for the proliferation, activation and antigen presentation of T cells, which are essential processes in the cell-mediated immune response against bacteria and viruses (Kafrouni et al., 2003; Grell, 1998). There- fore, blocking TNFR1-mediated signal transduction has emerged as a potential therapeutic strategy against inﬂammatory diseases with a low risk of side effects. Modelling the three-dimensional structure of the TNF–TNFR complex and understanding the differences between TNFR1 and TNFR2 will strongly contribute to the development of a TNFR1-selective therapeutic strategy.

To date, two crystal structures of TNFR1 have been reported: unliganded extracellular domains (PDB codes 1ncf and 1ext; Naismith et al., 1995, 1996) and a lymphotoxin  (LT-)–TNFR1 complex (PDB code 1tnr; Banner et al., 1993). The structure of wild- type TNF has also been reported (PDB code 1tnf; Eck & Sprang, 1989). Several structures of TNF mutants, including the TNF mutant used in this experiment, have also been reported (PDB codes 1a8m, 2az5, 4tsv, 5tsw, 2e7a and 2zjc; Reed et al., 1997; Cha et al., 1998; Shibata et al., 2008; Mukai et al., 2009). However, the structure of TNFR2 has not yet been reported, although the structure of a viral protein with homology to TNFR2 has been solved (PDB code 2uwi; Graham, 2007). This hinders the design of a structure-based drug for advanced anti-TNF therapy. Here, we describe the forma- tion, crystallization and preliminary X-ray diffraction analysis of the TNF–TNFR2 complex as part of our studies to determine the novel TNFR2 structure and aid in the development of TNFR1-selective therapies.



## 2. Methods

2.1. Preparation of TNF and TNFR2 The TNF molecule used in this experiment was a Lys-deﬁcient TNF mutant [mutTNF Lys( and K128P] created from wild-type TNF (wtTNF; NCBI accession No. X01394; Uniprot reference P01375) using a phage display system. This mutant retained full bioactivity and afﬁnity for TNFR, as described previously (Yamamoto et al., 2003). The wtTNF–TNFR2 complex formed efﬁciently, similar to that of mutTNF  $\mathrm{Lys}(-)$  and TNFR2, but did not crystallize under the conditions described below. MutTNF  $\mathrm{Lys}(-)$  (residues 77–233 in Uniprot P01375; 157 amino acids, without tags) was expressed in Escherichia coli and refolded from inclusion bodies as described previously (Yamamoto et al., 2003). Brieﬂy, TNF was produced in  $E.$ . coli BL21 (DE3) as inclusion bodies, which were washed in a buffer containing Triton X-100 and solubilized in  $6\;M$ guanidine–HCl,  $0.1\ M$  Tris–HCl  $\mathrm{pH}\,8.0\$  and  $2\:\mathrm{m}M$  EDTA. Solubilized protein at  $10\;\mathrm{mg}\;\mathrm{ml}^{-1}$  was reduced with  $65\;\mathrm{m}M$  dithioerythritol and refolded by a 100-fold dilution in a refolding buffer ( $.100\;\mathrm{m}M$  Tris–HCl,  $2\:\mathrm{m}M$  EDTA,  $0.5~M$  arginine and  $0.9\,\mathrm{m}M$  oxidized glutathione). After dialysis against  $20\;\mathrm{m}M$  Tris–HCl  $\mathrm{pH}~7.4\$  containing  $100\;\mathrm{m}M$  urea, active trimeric proteins were puriﬁed by ion-exchange chromatography using Q-Sepharose FF (GE Health- care Ltd). Size-exclusion chromatography was performed using a Superose 12 column (GE Healthcare) equilibrated with $20\;\mathrm{m}M$  Tris– HCl pH 7.4. The molecular mass of the puriﬁed TNF trimer was  $51\;\mathrm{kDa}$ . The complete sequence of this TNF molecule was as follows: VRSSSRTPSDMPVAHVVANPQAEGQLQWLNRRANALLANG- VELRDNQLVVPSEGLYLIYSQVLFSGQGCPSTHVLLTHTISRI- AVSYQTPVNLLSAIRSPCQRETPEGAEANPWYEPIYLGGVFQ- LEPGDRLSAEINRPDYLDFAESGQVYFGIIAL (residues mutated from wtTNF are shown in bold).

 



The extracellular domain of TNFR2 was used in this experiment (soluble TNFR2; NCBI accession No. M32315; Uniprot reference P20333). Puriﬁed soluble TNFR2 (residues 33–205 in Uniprot P20333; 173 amino acids, without tags) expressed in  $E.$ . coli was purchased from PeproTech Inc. The molecular mass was  $19\,\mathrm{kDa}$  as a monomer. The complete sequence of this TNFR2 molecule was as follows: APEPGSTCRLREYYDQTAQMCCSKCSPGQHAKVFC- TKTSDTVCDSCEDSTYTQLWNWVPECLSCGSRCSSDQVETQ- ACTREQNRICTCRPGWYCALSKQEGCRLCAPLRKCRPGFG- VARPGTETSDVVCKPCAPGTFSNTTSSTDICRPHQICNVVAIP- GNASMDAVCTSTSP. Lyophilized TNFR2 was dissolved in  $20\;\mathrm{m}M$  Tris–HCl pH 7.4 and used directly for complex formation.

2.2. Purification of the TNF–TNFR2 complex It is known that one TNF trimer binds three TNFR2 monomers and forms the TNF–TNFR2 complex. For crystallization of the complex, we attempted to purify the full  $110\;\mathrm{kDa}$  TNF–TNFR2 complex (one  $51\;\mathrm{kDa}$  TNF trimer and three  $19\;\mathrm{kDa}$  soluble TNFR2 monomers). The complex was formed by mixing TNFand TNFR2 for  $30\;\mathrm{min}$  at room temperature. Complex solutions with various molar ratios of TNF and TNFR2 were prepared and the molecular masses were assessed by analytical gel-ﬁltration chromatography using a tandem Superdex 75 10/300 GL/Superdex $200\,10/300\,\mathrm{GL}$  column (GE Healthcare) in  $20\;\mathrm{m}M$  Tris–HCl pH 7.4 (Fig. 1). The use of two gel- ﬁltration columns with different exclusion limits was useful for separating the complex from uncomplexed TNF and TNFR2. In this analysis, low-molecular-weight and high-molecular-weight gel- ﬁltration calibration kits (GE Healthcare) were used as protein standards. An increase in the mixing ratio of TNFR2 induced a shift in the retention peak of TNF to a higher molecular weight. At a TNF:TNFR2 ratio of 1:3 (Fig. 1e), a small peak at 57 min elution time corresponding to unbound  $19\;\mathrm{kDa}$  TNFR2 was observed. This peak containing excess TNFR2 was also observed at a 1:4 ratio (Fig.  $1f)$ . This result indicated that the full  $110\;\mathrm{kDa}$  TNF–TNFR2 complex was formed under these conditions. Therefore, ﬁnal fractionation was performed between 41 and  $44\,\mathrm{min}$  at a TNF:TNFR2 ratio of 1:4. The complex formation in the puriﬁed fraction was also conﬁrmed by SDS–PAGE (Fig. 2). This full TNF–TNFR2 complex was concen- trated to  $10\;\mathrm{mg}\;\mathrm{ml}^{-1}$  in  $20\;\mathrm{m}M$  Tris–HCl  $\mathrm{pH}~7.4\,\$  and used for crys- tallization as described below. The protein concentration was determined using Coomassie Protein Assay Reagent (Thermo Fisher Scientiﬁc Inc.).

 



2.3. Crystallization of the TNF–TNFR2 complex The sparse-matrix screening kit Crystal Screen HT and the systematic salt and PEG screening kit Index HT (Hampton Research, Aliso Viejo, California, USA) were used for initial crystallization trials using the sitting-drop vapour-diffusion method. Drops con- sisting of  $0.5~\upmu\mathrm{l}$  protein solution  $(10\;\mathrm{mg}\;\mathrm{ml}^{-1})$ ) and  $0.5~\upmu\mathrm{l}$  reservoir solution were equilibrated against  $100~\upmu\mathrm{l}$  reservoir solution in a CrystalEX 96-well ﬂat-bottom plate (Corning Incorporated, Corning, New York, USA). Two temperatures, 293 and  $277\;\mathrm{K}$ , were tested. Microcrystals were obtained using reservoir solution containing  PEG 3350 and  $0.2\;M$  $20\%(\omega/\nu)$  sodium formate at  $293\;\mathrm{K}$ . A grid screen around these conditions was evaluated using the hanging-drop  $1\;\upmu\mathrm{l}$  protein solution and  $1\;\upmu\mathrm{l}$  reservoir solution was equilibrated against  reservoir solution in a 24-well VDX Plate (Hampton $250\,\upmu\mathrm{l}$  Research). The crystallization conditions were improved using an additive screening kit (Hampton Research) and round-edged plate- shaped crystals were obtained from a droplet consisting of  $1\;\upmu\mathrm{l}$    $5\;\mathrm{mg}\;\mathrm{ml}^{-1}$ protein solution and  $1~\upmu\mathrm{l}$  reservoir solution containing  PEG 3350,  $8\%\left(w/\nu\right)$  $0.2\;M$  sodium formate and  $0.02\;M$  cobalt(II) chloride as an additive reagent. Protein and additive concentrations were optimized and the best crystals were obtained using  $7.5\,\upmu\mathrm{l}$    $0.8\;\mathrm{mg\;ml}^{-1}$ protein solution and  $7.5~\upmu\mathrm{l}$  reservoir solution containing  PEG 3350,  $7.5\%(w/\nu)$  $0.2\;M$  sodium formate and  $0.06\;M$  cobalt $(\mathrm{II})$  chloride. The crystals used for data collection had typical dimensions of  $0.2\,\times\,0.1\,\times\,0.03\;\mathrm{mm}$  (Fig. 3).



† $\begin{array}{r}{R_{\mathrm{merge}}=\sum_{h k l}\sum_{i}|I_{i}(h k l)-\langle I(h k l)\rangle|/{\sum_{h k l}\sum_{i}I_{i}(h k l)}}\end{array}$  hkl Pi jIiðhkl $h k l$ hIðhklÞij=P hkl P i IiðhklÞ, where  $I_{i}(h k l)$  is the ith intensity measurement of reﬂection, including symmetry-related reﬂections, and  $\langle I(h k l)\rangle$  is its average.

2.4. Preliminary X-ray analysis X-ray diffraction experiments were performed on BL41XU of the SPring-8 Synchrotron Facility in Harima, Japan. An ADSC Quantum 315 X-ray CCD detector was installed on BL41XU and used to record the diffraction data. Crystals were soaked in a cryoprotectant solution composed of  $7.5\%(\omega/\nu)$  PEG 3350, 0.2 M sodium formate,  cobalt(II) chloride and  $30\%$  glycerol. The crystals were then $0.06\;M$  mounted in nylon loops (Hampton Research) and ﬂash-cooled in a nitrogen stream at  $95~\mathrm{K}$ . Diffraction data were collected in  $1.0^{\circ}$  oscillation steps (180 images,  $0–180^{\circ}\rangle$  and the crystal diffracted X-rays to  $2.95\textrm{\AA}$ resolution. The data set was processed and scaled using the  program (Otwinowski & Minor, 1997). The crystals $H K L\mathrm{-}2000$  belonged to space group  $P2_{1}2_{1}2_{1}$ , with unit-cell parameters  $a=74.5$ , , $b=117.4$   $c=\bar{2}46.8\;\mathring\mathrm{A}$ . Data-collection statistics are shown in Table 1.

 



## 3. Results and discussion

The results of SDS–PAGE using the crystallization solution suggested that the crystal contained both TNF and TNFR2. Gel-ﬁltration analysis showed that the TNF–TNFR2 complex with a molecular mass of  $110\;\mathrm{kDa}$  contained both a TNF trimer  $(51\;\mathrm{kDa})$  and three TNFR2 monomers  $(3\,\times\,19\;\mathrm{kDa})$ ). This suggests that the asymmetric unit of the crystals contained two molecules of the TNF–TNFR2 complex (two TNF trimers and six TNFR2 monomers). The calcu- lated solvent content was  $50.7\%$  and the Matthews coefﬁcient was   $2.49\;\mathring{\mathrm{A}}^{3}\;\mathrm{Da}^{-1}$ (Matthews, 1968).

Molecular replacement was performed with the MOLREP pro- gram (Vagin & Teplyakov, 1997) in4i (Potterton et al., 2003) using the TNF mutant structure described in our previous report (PDB code 2e7a; Shibata, 2008) as a search model. Two signiﬁcant solutions corresponding to TNF trimers were obtained ( $(R=0.469)$ ). The model from molecular replacement was subjected to rigid-body reﬁnement, simulated annealing, energy minimization and -factor reﬁnement using $B$  CNS (Bru¨nger et al., 1998). The initial electron-density map calculated using only the phases of reﬁned TNF trimer structures ( $R=0.425$  and  $R_{\mathrm{free}}=0.493\AA$ ) clearly showed almost the entire TNF trimer. The main chains and side chains of TNFR2 could also be detected around the TNF trimer. Manual model building of TNFR2 based on the TNFR1 structure using the Coot program (Emsley & Cowtan, 2004) in  $C C P4i$  is in progress.

This study was supported by Research for Promoting Techno- logical Seeds (No. 11-067) from the Japan Science and Technology Agency (JST), Research Fund Project on Health Sciences focusing on Drug Innovation (No. KAA3701) from the Japan Health Sciences Foundation, grants from the Ministry of Health, Labor and Welfare in Japan, a Grant-in-Aid for Young Scientists (B) (No. 20790134) and Grants-in-Aid for Scientiﬁc Research (Nos. 17016084, 17689008, 17790135, 18015055, 18659047, 20015052 and 20200017) from the Ministry of Education, Culture, Sports, Science and Technology of Japan (MEXT) and Research Fellowships for Young Scientists (No. 20-3919) from Japan Society for the Promotion of Science (JSPS). This study was also supported by Hayashibara Biochemical Labora- tories Inc., Okayama, Japan.



## References

Aggarwal, B. B. (2003). 3, 745–756. Banner, D. W., D’Arcy, A., Janes, W., Gentz, R., Schoenfeld, H. J., Broger, C., Loetscher, H. & Lesslauer, W. (1993)., 73, 431–445. Bru¨nger, A. T., Adams, P. D., Clore, G. M., DeLano, W. L., Gros, P., Grosse- Kunstleve, R. W., Jiang, J.-S., Kuszewski, J., Nilges, M., Pannu, N. S., Read, 54, 905–921.

Cha, S. S., Kim, J. S., Cho, H. S., Shin, N. K., Jeong, W., Shin, H. C., Kim, Y. J.,  273  264 60  2  9 Graham, S. C., Bahar, M. W., Abrescia, N. G., Smith, G. L., Stuart, D. I. & Grimes, J. M. (2007). 372, 660–671. Grell, M., Becke, F. M., Wajant, H., Mannel, D. N. & Scheurich, P. (1998). J. Immunol. 28, 257–263. Kafrouni, M. I., Brown, G. R. & Thiele, D. L. (2003). 74, 564–571. Leist, M., Gantner, F., Jilg, S. & Wendel, A. (1995). 154, 1307– 1316. Matthews, B. W. (1968). 33, 491–497. Mori, L., Iselin, S., De Libero, G. & Lesslauer, W. (1996). 157, 3178–3182. Mukai, Y., Shibata, H., Nakamura, T., Yoshioka, Y., Abe, Y., Nomura, T., Taniai, M., Ohta, T., Ikemizu, S., Nakagawa, S., Tsunoda, S., Kamada, H., Yamagata, Y. & Tsutsumi, Y. (2009). 30, 1121–1129. Naismith, J. H., Devine, T. Q., Brandhuber, B. J. & Sprang, S. R. (1995). Chem. 270, 13303–13307. Naismith, J. H., Devine, T. Q., Kohno, T. & Sprang, S. R. (1996)., 4, 1251–1262. Otwinowski, Z. & Minor, W. (1997). 276, 307–326. Potterton, E., Briggs, P., Turkenburg, M. & Dodson, E. (2003). D59, 1131–1137. Reed, C., Fu, Z. Q., Wu, J., Xue, Y. N., Harrison, R. W., Chen, M. J. & Weber, I. T. (1997). 10 Shibata, H. 283  30 Yamamoto, Y., Tsutsumi, Y., Yoshioka, Y., Nishibata, T., Kobayashi, K., Okamoto, T., Mukai, Y., Shimizu, T., Nakagawa, S., Nagata, S. & Mayumi, T. (2003). 21


Acta Crystallographica Section F Structural Biology and Crystallization Communications

ISSN 1744-3091

Yohei Mukai, Teruya Nakamura, Yasuo Yoshioka, Shin-ichi Tsunoda, Haruhiko Kamada, Shinsaku Nakagawa, Yuriko Yamagata and Yasuo Tsutsumi\*

 



Graduate School of Pharmaceutical Sciences, Osaka University, Japan, bLaboratory of Pharmaceutical Proteomics, National Institute of Biomedical Innovation (NiBio), Japan, Graduate School of Pharmaceutical Sciences, Kumamoto University, Japan, and dThe Center of Advanced Medical Engineering and Informatics, Osaka University, Japan Correspondence e-mail: ytsutsumi@phs.osaka-u.ac.jp Received 29 September 2008 Accepted 7 February 2009

 



# Crystallization and preliminary X-ray analysis of the tumour necrosis factor–tumour necrosis factor receptor type 2 complex

Tumour necrosis factor receptor type 2 (TNFR2, TNFRSF1B) is an essential receptor for various host-defence functions of tumour necrosis factor  $\alpha$  (TNF). As part of studies to determine the structure of TNFR2, the formation, crystallization and preliminary  $X.$ -ray diffraction analysis of the TNF–TNFR2 complex are described. The TNF–TNFR2 complex, which comprises one TNF trimer and three TNFR2 monomers, was conﬁrmed and puriﬁed by size- exclusion chromatography. Crystals of the TNF–TNFR2 complex were obtained using polyethylene glycol 3350 as a precipitant. The crystal belonged to space group  $P2_{1}2_{1}2_{1}$ , with unit-cell parameters  $a\ =\ 74.5$ ,  $b\ =\ 117.4$ ,  $\mathrm{~\bar{~}{~c~}~}=\,246.8\:\mathrm{~\overset~{A}~{~}~}$ . Assuming the presence of two TNF–TNFR2 complexes in the asymmetric unit, the Matthews coefﬁcient  $V_{\mathbf{M}}$  was  $2.49\;\mathring{\mathrm{A}}^{3}\;\mathrm{Da}^{-1}$  and the solvent content of the crystal was  $50.7\%$ . The crystal diffracted to  $2.95\mathrm{~\AA~}$ resolution.

## 1. Introduction

Tumour necrosis factor  $\alpha$  (TNF) is a major inﬂammatory cytokine that plays a central role in host defence and inﬂammation (Aggarwal, 2003). Overexpression of TNF is closely associated with inﬂammatory diseases such as rheumatoid arthritis and inﬂammatory bowel disease (Feldmann & Maini, 2003). Therefore, anti-TNF antibodies and soluble TNF receptors (TNFR) that interfere with the activity of TNF have been used to treat various inﬂammatory diseases (Feldmann, 2002). However, anti-TNF therapy inhibits both types of TNFR (TNFR1 and TNFR2), often causing serious side effects.

Previous studies using animal models of diseases such as arthritis and hepatitis have demonstrated that TNFR1 plays a predominant role in the pathogenesis and exacerbation of inﬂammation (Mori et al., 1996; Leist et al., 1995). In contrast, TNFR2 is crucial for the proliferation, activation and antigen presentation of T cells, which are essential processes in the cell-mediated immune response against bacteria and viruses (Kafrouni et al., 2003; Grell, 1998). There- fore, blocking TNFR1-mediated signal transduction has emerged as a potential therapeutic strategy against inﬂammatory diseases with a low risk of side effects. Modelling the three-dimensional structure of the TNF–TNFR complex and understanding the differences between TNFR1 and TNFR2 will strongly contribute to the development of a TNFR1-selective therapeutic strategy.

To date, two crystal structures of TNFR1 have been reported: unliganded extracellular domains (PDB codes 1ncf and 1ext; Naismith et al., 1995, 1996) and a lymphotoxin  (LT-)–TNFR1 complex (PDB code 1tnr; Banner et al., 1993). The structure of wild- type TNF has also been reported (PDB code 1tnf; Eck & Sprang, 1989). Several structures of TNF mutants, including the TNF mutant used in this experiment, have also been reported (PDB codes 1a8m, 2az5, 4tsv, 5tsw, 2e7a and 2zjc; Reed et al., 1997; Cha et al., 1998; Shibata et al., 2008; Mukai et al., 2009). However, the structure of TNFR2 has not yet been reported, although the structure of a viral protein with homology to TNFR2 has been solved (PDB code 2uwi; Graham, 2007). This hinders the design of a structure-based drug for advanced anti-TNF therapy. Here, we describe the forma- tion, crystallization and preliminary X-ray diffraction analysis of the TNF–TNFR2 complex as part of our studies to determine the novel TNFR2 structure and aid in the development of TNFR1-selective therapies.



## 2. Methods

2.1. Preparation of TNF and TNFR2 The TNF molecule used in this experiment was a Lys-deﬁcient TNF mutant [mutTNF Lys( and K128P] created from wild-type TNF (wtTNF; NCBI accession No. X01394; Uniprot reference P01375) using a phage display system. This mutant retained full bioactivity and afﬁnity for TNFR, as described previously (Yamamoto et al., 2003). The wtTNF–TNFR2 complex formed efﬁciently, similar to that of mutTNF  $\mathrm{Lys}(-)$  and TNFR2, but did not crystallize under the conditions described below. MutTNF  $\mathrm{Lys}(-)$  (residues 77–233 in Uniprot P01375; 157 amino acids, without tags) was expressed in Escherichia coli and refolded from inclusion bodies as described previously (Yamamoto et al., 2003). Brieﬂy, TNF was produced in  $E.$ . coli BL21 (DE3) as inclusion bodies, which were washed in a buffer containing Triton X-100 and solubilized in  $6\;M$ guanidine–HCl,  $0.1\ M$  Tris–HCl  $\mathrm{pH}\,8.0\$  and  $2\:\mathrm{m}M$  EDTA. Solubilized protein at  $10\;\mathrm{mg}\;\mathrm{ml}^{-1}$  was reduced with  $65\;\mathrm{m}M$  dithioerythritol and refolded by a 100-fold dilution in a refolding buffer ( $.100\;\mathrm{m}M$  Tris–HCl,  $2\:\mathrm{m}M$  EDTA,  $0.5~M$  arginine and  $0.9\,\mathrm{m}M$  oxidized glutathione). After dialysis against  $20\;\mathrm{m}M$  Tris–HCl  $\mathrm{pH}~7.4\$  containing  $100\;\mathrm{m}M$  urea, active trimeric proteins were puriﬁed by ion-exchange chromatography using Q-Sepharose FF (GE Health- care Ltd). Size-exclusion chromatography was performed using a Superose 12 column (GE Healthcare) equilibrated with $20\;\mathrm{m}M$  Tris– HCl pH 7.4. The molecular mass of the puriﬁed TNF trimer was  $51\;\mathrm{kDa}$ . The complete sequence of this TNF molecule was as follows: VRSSSRTPSDMPVAHVVANPQAEGQLQWLNRRANALLANG- VELRDNQLVVPSEGLYLIYSQVLFSGQGCPSTHVLLTHTISRI- AVSYQTPVNLLSAIRSPCQRETPEGAEANPWYEPIYLGGVFQ- LEPGDRLSAEINRPDYLDFAESGQVYFGIIAL (residues mutated from wtTNF are shown in bold).

 



The extracellular domain of TNFR2 was used in this experiment (soluble TNFR2; NCBI accession No. M32315; Uniprot reference P20333). Puriﬁed soluble TNFR2 (residues 33–205 in Uniprot P20333; 173 amino acids, without tags) expressed in  $E.$ . coli was purchased from PeproTech Inc. The molecular mass was  $19\,\mathrm{kDa}$  as a monomer. The complete sequence of this TNFR2 molecule was as follows: APEPGSTCRLREYYDQTAQMCCSKCSPGQHAKVFC- TKTSDTVCDSCEDSTYTQLWNWVPECLSCGSRCSSDQVETQ- ACTREQNRICTCRPGWYCALSKQEGCRLCAPLRKCRPGFG- VARPGTETSDVVCKPCAPGTFSNTTSSTDICRPHQICNVVAIP- GNASMDAVCTSTSP. Lyophilized TNFR2 was dissolved in  $20\;\mathrm{m}M$  Tris–HCl pH 7.4 and used directly for complex formation.

2.2. Purification of the TNF–TNFR2 complex It is known that one TNF trimer binds three TNFR2 monomers and forms the TNF–TNFR2 complex. For crystallization of the complex, we attempted to purify the full  $110\;\mathrm{kDa}$  TNF–TNFR2 complex (one  $51\;\mathrm{kDa}$  TNF trimer and three  $19\;\mathrm{kDa}$  soluble TNFR2 monomers). The complex was formed by mixing TNFand TNFR2 for  $30\;\mathrm{min}$  at room temperature. Complex solutions with various molar ratios of TNF and TNFR2 were prepared and the molecular masses were assessed by analytical gel-ﬁltration chromatography using a tandem Superdex 75 10/300 GL/Superdex $200\,10/300\,\mathrm{GL}$  column (GE Healthcare) in  $20\;\mathrm{m}M$  Tris–HCl pH 7.4 (Fig. 1). The use of two gel- ﬁltration columns with different exclusion limits was useful for separating the complex from uncomplexed TNF and TNFR2. In this analysis, low-molecular-weight and high-molecular-weight gel- ﬁltration calibration kits (GE Healthcare) were used as protein standards. An increase in the mixing ratio of TNFR2 induced a shift in the retention peak of TNF to a higher molecular weight. At a TNF:TNFR2 ratio of 1:3 (Fig. 1e), a small peak at 57 min elution time corresponding to unbound  $19\;\mathrm{kDa}$  TNFR2 was observed. This peak containing excess TNFR2 was also observed at a 1:4 ratio (Fig.  $1f)$ . This result indicated that the full  $110\;\mathrm{kDa}$  TNF–TNFR2 complex was formed under these conditions. Therefore, ﬁnal fractionation was performed between 41 and  $44\,\mathrm{min}$  at a TNF:TNFR2 ratio of 1:4. The complex formation in the puriﬁed fraction was also conﬁrmed by SDS–PAGE (Fig. 2). This full TNF–TNFR2 complex was concen- trated to  $10\;\mathrm{mg}\;\mathrm{ml}^{-1}$  in  $20\;\mathrm{m}M$  Tris–HCl  $\mathrm{pH}~7.4\,\$  and used for crys- tallization as described below. The protein concentration was determined using Coomassie Protein Assay Reagent (Thermo Fisher Scientiﬁc Inc.).

 



2.3. Crystallization of the TNF–TNFR2 complex The sparse-matrix screening kit Crystal Screen HT and the systematic salt and PEG screening kit Index HT (Hampton Research, Aliso Viejo, California, USA) were used for initial crystallization trials using the sitting-drop vapour-diffusion method. Drops con- sisting of  $0.5~\upmu\mathrm{l}$  protein solution  $(10\;\mathrm{mg}\;\mathrm{ml}^{-1})$ ) and  $0.5~\upmu\mathrm{l}$  reservoir solution were equilibrated against  $100~\upmu\mathrm{l}$  reservoir solution in a CrystalEX 96-well ﬂat-bottom plate (Corning Incorporated, Corning, New York, USA). Two temperatures, 293 and  $277\;\mathrm{K}$ , were tested. Microcrystals were obtained using reservoir solution containing  PEG 3350 and  $0.2\;M$  $20\%(\omega/\nu)$  sodium formate at  $293\;\mathrm{K}$ . A grid screen around these conditions was evaluated using the hanging-drop  $1\;\upmu\mathrm{l}$  protein solution and  $1\;\upmu\mathrm{l}$  reservoir solution was equilibrated against  reservoir solution in a 24-well VDX Plate (Hampton $250\,\upmu\mathrm{l}$  Research). The crystallization conditions were improved using an additive screening kit (Hampton Research) and round-edged plate- shaped crystals were obtained from a droplet consisting of  $1\;\upmu\mathrm{l}$    $5\;\mathrm{mg}\;\mathrm{ml}^{-1}$ protein solution and  $1~\upmu\mathrm{l}$  reservoir solution containing  PEG 3350,  $8\%\left(w/\nu\right)$  $0.2\;M$  sodium formate and  $0.02\;M$  cobalt(II) chloride as an additive reagent. Protein and additive concentrations were optimized and the best crystals were obtained using  $7.5\,\upmu\mathrm{l}$    $0.8\;\mathrm{mg\;ml}^{-1}$ protein solution and  $7.5~\upmu\mathrm{l}$  reservoir solution containing  PEG 3350,  $7.5\%(w/\nu)$  $0.2\;M$  sodium formate and  $0.06\;M$  cobalt $(\mathrm{II})$  chloride. The crystals used for data collection had typical dimensions of  $0.2\,\times\,0.1\,\times\,0.03\;\mathrm{mm}$  (Fig. 3).



† $\begin{array}{r}{R_{\mathrm{merge}}=\sum_{h k l}\sum_{i}|I_{i}(h k l)-\langle I(h k l)\rangle|/{\sum_{h k l}\sum_{i}I_{i}(h k l)}}\end{array}$  hkl Pi jIiðhkl $h k l$ hIðhklÞij=P hkl P i IiðhklÞ, where  $I_{i}(h k l)$  is the ith intensity measurement of reﬂection, including symmetry-related reﬂections, and  $\langle I(h k l)\rangle$  is its average.

2.4. Preliminary X-ray analysis X-ray diffraction experiments were performed on BL41XU of the SPring-8 Synchrotron Facility in Harima, Japan. An ADSC Quantum 315 X-ray CCD detector was installed on BL41XU and used to record the diffraction data. Crystals were soaked in a cryoprotectant solution composed of  $7.5\%(\omega/\nu)$  PEG 3350, 0.2 M sodium formate,  cobalt(II) chloride and  $30\%$  glycerol. The crystals were then $0.06\;M$  mounted in nylon loops (Hampton Research) and ﬂash-cooled in a nitrogen stream at  $95~\mathrm{K}$ . Diffraction data were collected in  $1.0^{\circ}$  oscillation steps (180 images,  $0–180^{\circ}\rangle$  and the crystal diffracted X-rays to  $2.95\textrm{\AA}$ resolution. The data set was processed and scaled using the  program (Otwinowski & Minor, 1997). The crystals $H K L\mathrm{-}2000$  belonged to space group  $P2_{1}2_{1}2_{1}$ , with unit-cell parameters  $a=74.5$ , , $b=117.4$   $c=\bar{2}46.8\;\mathring\mathrm{A}$ . Data-collection statistics are shown in Table 1.

 



## 3. Results and discussion

The results of SDS–PAGE using the crystallization solution suggested that the crystal contained both TNF and TNFR2. Gel-ﬁltration analysis showed that the TNF–TNFR2 complex with a molecular mass of  $110\;\mathrm{kDa}$  contained both a TNF trimer  $(51\;\mathrm{kDa})$  and three TNFR2 monomers  $(3\,\times\,19\;\mathrm{kDa})$ ). This suggests that the asymmetric unit of the crystals contained two molecules of the TNF–TNFR2 complex (two TNF trimers and six TNFR2 monomers). The calcu- lated solvent content was  $50.7\%$  and the Matthews coefﬁcient was   $2.49\;\mathring{\mathrm{A}}^{3}\;\mathrm{Da}^{-1}$ (Matthews, 1968).

Molecular replacement was performed with the MOLREP pro- gram (Vagin & Teplyakov, 1997) in4i (Potterton et al., 2003) using the TNF mutant structure described in our previous report (PDB code 2e7a; Shibata, 2008) as a search model. Two signiﬁcant solutions corresponding to TNF trimers were obtained ( $(R=0.469)$ ). The model from molecular replacement was subjected to rigid-body reﬁnement, simulated annealing, energy minimization and -factor reﬁnement using $B$  CNS (Bru¨nger et al., 1998). The initial electron-density map calculated using only the phases of reﬁned TNF trimer structures ( $R=0.425$  and  $R_{\mathrm{free}}=0.493\AA$ ) clearly showed almost the entire TNF trimer. The main chains and side chains of TNFR2 could also be detected around the TNF trimer. Manual model building of TNFR2 based on the TNFR1 structure using the Coot program (Emsley & Cowtan, 2004) in  $C C P4i$  is in progress.

This study was supported by Research for Promoting Techno- logical Seeds (No. 11-067) from the Japan Science and Technology Agency (JST), Research Fund Project on Health Sciences focusing on Drug Innovation (No. KAA3701) from the Japan Health Sciences Foundation, grants from the Ministry of Health, Labor and Welfare in Japan, a Grant-in-Aid for Young Scientists (B) (No. 20790134) and Grants-in-Aid for Scientiﬁc Research (Nos. 17016084, 17689008, 17790135, 18015055, 18659047, 20015052 and 20200017) from the Ministry of Education, Culture, Sports, Science and Technology of Japan (MEXT) and Research Fellowships for Young Scientists (No. 20-3919) from Japan Society for the Promotion of Science (JSPS). This study was also supported by Hayashibara Biochemical Labora- tories Inc., Okayama, Japan.



## References

Aggarwal, B. B. (2003). 3, 745–756. Banner, D. W., D’Arcy, A., Janes, W., Gentz, R., Schoenfeld, H. J., Broger, C., Loetscher, H. & Lesslauer, W. (1993)., 73, 431–445. Bru¨nger, A. T., Adams, P. D., Clore, G. M., DeLano, W. L., Gros, P., Grosse- Kunstleve, R. W., Jiang, J.-S., Kuszewski, J., Nilges, M., Pannu, N. S., Read, 54, 905–921.

Cha, S. S., Kim, J. S., Cho, H. S., Shin, N. K., Jeong, W., Shin, H. C., Kim, Y. J.,  273  264 60  2  9 Graham, S. C., Bahar, M. W., Abrescia, N. G., Smith, G. L., Stuart, D. I. & Grimes, J. M. (2007). 372, 660–671. Grell, M., Becke, F. M., Wajant, H., Mannel, D. N. & Scheurich, P. (1998). J. Immunol. 28, 257–263. Kafrouni, M. I., Brown, G. R. & Thiele, D. L. (2003). 74, 564–571. Leist, M., Gantner, F., Jilg, S. & Wendel, A. (1995). 154, 1307– 1316. Matthews, B. W. (1968). 33, 491–497. Mori, L., Iselin, S., De Libero, G. & Lesslauer, W. (1996). 157, 3178–3182. Mukai, Y., Shibata, H., Nakamura, T., Yoshioka, Y., Abe, Y., Nomura, T., Taniai, M., Ohta, T., Ikemizu, S., Nakagawa, S., Tsunoda, S., Kamada, H., Yamagata, Y. & Tsutsumi, Y. (2009). 30, 1121–1129. Naismith, J. H., Devine, T. Q., Brandhuber, B. J. & Sprang, S. R. (1995). Chem. 270, 13303–13307. Naismith, J. H., Devine, T. Q., Kohno, T. & Sprang, S. R. (1996)., 4, 1251–1262. Otwinowski, Z. & Minor, W. (1997). 276, 307–326. Potterton, E., Briggs, P., Turkenburg, M. & Dodson, E. (2003). D59, 1131–1137. Reed, C., Fu, Z. Q., Wu, J., Xue, Y. N., Harrison, R. W., Chen, M. J. & Weber, I. T. (1997). 10 Shibata, H. 283  30 Yamamoto, Y., Tsutsumi, Y., Yoshioka, Y., Nishibata, T., Kobayashi, K., Okamoto, T., Mukai, Y., Shimizu, T., Nakagawa, S., Nagata, S. & Mayumi, T. (2003). 21


Molecular Determinants of HIV-2 R5–X4 Tropism in the V3 Loop: Development of a New Genotypic Tool

### Benoit Visseaux, Margarita Hurtado-Nedelec, Charlotte Charpentier,1,4 Gilles Collin,1,4 Alexandre Storto,1,4

Sophie Matheron, Lucile Larrouy, Florence Damond, Francxoise Brun-Ve´zinet, Diane Descamps, and the ANRS CO 05 HIV-2 Cohort Laboratoire de Virologie,De´partement d'He´matologie et Immunologie, and 3Service des Maladies Infectieuses et Tropicales, Assistance Publique-Hoˆpitaux de Paris, Hoˆpital Bichat-Claude Bernard, and $^{4}\mathsf{E A}$  4409 Paris-Diderot University, France



Objective.The use of CCR5 inhibitors requires a tool to predict human immunodeﬁciency virus type 2 (HIV-2) tropism, as established in HIV-1. The aim of our study was to identify genotypic determinants of HIV-2 tropism located in the gp105 V3 loop.

Methods.HIV-2 tropism phenotypic assays were performed on 53 HIV-2 clinical isolates using GFP expressing human osteosarcoma T4 [GHOST(3)] cell lines expressing CD4 and CCR5 or CXCR4 coreceptors. The gp105 V3 loop was sequenced and analyzed.

Results.Thirty-four HIV-2 isolates were classiﬁed as R5, 7 as X4, and 12 as X4/R5 (dual). Substitution at residue 18 was always associated with a dual/X4 tropism ( $P<.00001)$ . The following determinants were associated with dual/X4 tropism: a global net charge of more than  $^{+6}$   $(P<.00001)$ , V19K/R mutation ( $\left(P<.00001\right)$ ), S22A/F/Y mutation ( $(P<.002)$ ), Q23R mutation ( $P<.00001)$ , and insertions at residue 24 ( $P<.00001)$ ), I25L/Y ( $P<.0004)$ ), R28K ( $P<.0004)$ , and R30K ( $P<.014)$ ). These mutations were not found in R5 isolates, except R28K and R30K, which were detected in 4 and 5 R5 isolates, respectively. The 4 major genotypic determinants of dual/X4 tropism were mutation at residue 18,  $\mathrm{V}19~\mathrm{K}/\mathrm{R}$  mutation, insertions at residue 24, and V3 global net charge.

Conclusions.We established a strong association between HIV-2 phenotypic tropism and V3-loop sequences, allowing for the prediction of R5- and/or X4-tropic viruses in HIV-2 infection.

The human immunodeﬁciency virus (HIV) is character- ized by a high level of genetic diversity within HIV type 1 (HIV-1) and HIV type 2 (HIV-2). HIV-2 infections are mainly restricted to West Africa, including Guinea-Bissau, Gambia, Senegal, and Guinea. Some European countries are also concerned about HIV-2 infection, which repre- sents  $5\%$  of HIV infection in a series of patients in Por- tugal [1] and  $2\%$  of the new HIV infections in France [2].

HIV-2 clinical isolates appear to be susceptible to nu- cleoside reverse-transcriptase inhibitors and to the new drug class of integrase inhibitors [3, 4]. However, they are naturally resistant to nonnucleoside reverse-transcriptase inhibitors [5, 6] and to fusion-inhibitor drug classes [6, 7]. In addition, HIV-2 shows decreased susceptibility to some protease inhibitors, such as amprenavir, atazanavir, and tipranavir [6, 8]. In this context, the new class of CCR5 coreceptor (R5) inhibitors with maraviroc may provide a new therapeutic opportunity in HIV-2 infection man- agement. However, few data are available on maraviroc use in HIV-2 infection. One study found in vitro antiviral activity of nonlicensed CCR5 inhibitors on HIV-2 [9] and 2 HIV-2–infected patients with a multiresistant virus showed an early virological response after receiving a multiple-drug salvage regimen with maraviroc [10, 11]. The process of HIV-2 entry involves identical steps to HIV-1, including similar virus-cell interactions, ﬁrst be- tween the HIV-2 gp105 envelope glycoprotein and CD4 receptor and then between the gp105 V3 loop and CCR5 or CXCR4 coreceptors [12, 13]. Furthermore, HIV-2 seems to be able to use in vitro a broad range of coreceptors, such as CCR1, CCR2b, CCR3, CCR8, BOB/GPR15, or BONZO/CXCR6 [12, 14–17]. However, discrepancies exist between studies and cell lines models for tropism determination. Indeed, U87 cell lines showed higher level of nonspeciﬁc infection than GFP expressing human osteosarcoma T4 [GHOST(3)] cell lines [17, 18]. In the GHOST(3) cell line model, only CCR5, GPR15, and CXCR6 were efﬁciently used in vitro by R5-tropic HIV-2 viruses, and only CXCR4 was efﬁciently used by X4-tropic HIV-2 virus [18]. Moreover, 2 studies assessed in vitro the relative contribution of these different coreceptors in HIV-2 infection using coreceptor- speciﬁcinhibitorsonperipheral-bloodmononuclearcells (PBMCs) or a PBMC model with  $\mathrm{CCR5^{+/+}}$ and  $\mathrm{CCR5^{-/-}}$  blood donors, showing that only CCR5- or CXCR4-dependent infection is really efﬁcient on PBMC [13, 19].



CCR5 inhibitors are ineffective against the CXCR4 (X4)–using virus [20, 21]. Thus, it is mandatory to assess viral tropism in order to determine a patient’s eligibility for maraviroc therapy. In HIV-1 infection, tropism can be determined by a phenotypic recombi- nant test (Troﬁle; Monogram Biosciences) [22] or by a genotypic test that analyzes the gp120 V3-loop sequences [23, 24]. Genotypic interpretation of V3-loop sequences for tropism determination can be performed using different algorithms [25, 26] that take into account the major genotypic determinants of coreceptors used in the HIV-1 genetic context [27, 28].

In HIV-2 infection, several studies have evaluated the corre- lations between the gp105 V3-loop sequence and HIV-2 tropism, as assessed by phenotypic tests [15–17, 29–31]. All these studies have reported a few HIV-2 strains, and have reached discordant conclusions. The aim of this study was to establish a genotypic tool for HIV-2-tropism determination, as a ﬁrst step in studying tropism implications in HIV-2 infection.

## MATERIALS AND METHODS

Study Patients and Virus Stocks Fifty-three HIV-2 clinical isolates, from 53 HIV-2–infected, maraviroc-naive patients of the French Agence Nationale de Re- cherches sur le SIDA et les he´patites virales (ANRS) HIV-2 cohort  $(\mathrm{CO}~05)$ , were isolated by cocultivation of phytohemagglutinin- activated PBMCs from a pool of healthy blood donors, as de- scribed elsewhere [3]. Written consent was provided by all patients at the time of inclusion in the French ANRS CO 05 HIV-2 cohort. Among the 53 patients, 37  $(70\%)$  were infected with HIV-2 group A, 15( $(28\%)$  with HIV-2 groupB, and1 with HIV-2groupH  $(2\%)$ . The HIV-2 group A reference strain,  $\mathrm{HV-}2_{\mathrm{ROD}},$  using both CCR5 and CXCR4 coreceptors, was included in our series.

Cell Lines Peripheral-blood mononuclear cells were obtained from HIV- seronegativeblooddonors.Theywerestimulatedwith phytohemagglutinin and cultured, as described elsewhere [3]. GHOST(3) cell lines are human osteosarcoma cells engineered to stably express the CD4 receptor and one HIV coreceptor, CCR5 or CXCR4. Both GHOST(3) cell lines (CD4 plus CCR5 or CD4 plus CXCR4) also carry a humanized green ﬂuorescent protein (GFP) gene under the control of a HIV-2 long terminal repeat [15]. Infection with HIV and consecutive production of the Tat protein induces expression of GFP, which can be detected by ﬂuorescent microscopy or cytometric analysis. GHOST(3) cell lines were cultured in Dulbecco’s Modiﬁed Eagle Medium (DMEM) medium (Invitrogen) containing  $10\%$  fetal bovine , $(100~\upmu\mathrm{g/mL})$  $(500~\upmu\mathrm{g/mL})$  and puromycin  $(1~\upmu\mathrm{g/mL})$ ). GHOST(3) High CCR5 coreceptors (Hi-5) cells and GHOST(3) CXCR4 cells were provided by the National Institutes of Health (NIH) AIDS Reagent Program. At the time of this work, parental cell lines expressing only the CD4 receptor without any coreceptor were not available and were still not available at the writing of this work.



Phenotypic Tropism Assay GHOST(3) cells were seeded in 24-well plates at  $40\;000$  per well in growth medium and incubated for 24 hours at  $37^{\circ}\mathrm{C}$  in  $5\%$  carbon dioxide  $(\mathrm{CO}_{2})$ . After incubation, the culture medium was removed from the wells, and  $300~\upmu\mathrm{L}$  of cell-free virus, at a titer between  $10^{7}$  and  $10^{8}\;\mathrm{copies/mL}$  $10^{8}$  and contained in DMEM, with  $10\%$  FBS and  polybrene, was added to the cells. Plates were immediately $7.5\,\upmu\mathrm{g/L}$  centrifuged for 30 minutes at  $2300\ g.$ . The centrifugation step and the addition of polybrene, an anionic polymer that enhances virus- cell contacts by shielding charge and virus-aggregation mecha- nisms, were added to increase infection efﬁciency [32–34]. Plates were incubated for 4 hours at  $37^{\circ}\mathrm{C}$  in  $5\%\ \mathrm{CO}_{2}$ . The viral super- natant was then removed, and  $1\;\mathrm{mL}$  of culture medium was added to each well. Plates were incubated for 3 days at  $37^{\circ}\mathrm{C}$  in  $5\%\;\mathrm{CO}_{2}$ . Cells were then observed for cytopathogenic effects, the culture  of trypsin was added. After $90~\upmu\mathrm{L}$  5 minutes of incubation at  $37^{\circ}\mathrm{C},$  trypsin was inactivated by the addition of  $300~\upmu\mathrm{L}$  of phosphate-buffered saline and  $20\%$  FBS. Cells were analyzed within an hour by ﬂow cytometry, after staining with propidium iodide in order to assess cellular viability. A negative control, containing only culture medium, was included in each experiment. The ratio-to-cell-negative (RTCN) value was used to quantify cell infection [18, 35]. The RTCN value is the ratio of the percentage of ﬂuorescence-positive cells to their mean ﬂuorescence intensity (FI), calculated as follows:  $\mathrm{RTCN}=(\%$  of GFP-positive cells) of infected cells/( $\%$  of GFP-positive cells $\times\mathrm{{FI}}_{\ell}$   of uninfected cells. A RTCN value of 5 was taken as the $\times\mathrm{\FI})$  negative cutoff threshold. In our series, when the proportion of GFP-positive cells was  ${<}10\%$ , the phenotypic tropism assay was performed again using a 2-fold larger viral inoculum. No dis- cordance was found with these controls (data not shown).



GHOST(3) cell lines are known to endogenously express in low-level proportion the CXCR4 coreceptor [36, 37]. Thus, low levels of infection were described in previous study with HIV-2 X4 isolates in GHOST(3) cell lines [18]. In regard to these latter data, in our study all HIV-2 isolates with a CCR5 cell RTCN value  of the CXCR4 cell RTCN value were considered X4 tropic. ${<}15\%$  A very low-level infection of parental GHOST cell lines has been described elsewhere in some HIV-2 isolates, independently of CCR5 or CXCR4 coreceptors [18]. The proposed mechanism could be coreceptor-independent infection or alternative cor- eceptor usage. However, these infections are infrequent and have a very low efﬁciency. In regards to the data obtained by Blaak et al [18], all HIV-2 isolates that have a CXCR4 RTCN value  ${<}5\%$  of the CCR5 cell RTCN value were considered R5 tropic.



HIV-2 Gene Sequencing HIV-2 RNA was extracted from 1 mL of viral supernatant using a Total NA Large Volume kit (Roche Diagnostic) on a MagnaPure automate (Roche Diagnostic), according to the manufacturer’s recommendations. A 1000–base pair fragment of gp105, encom- passing the V3 loop, was ﬁrst ampliﬁed by reverse-transcription polymerase chain reaction (RT-PCR) according to the manu- facturer’s instructions (Titan One-Tube RT-PCR kit; Roche Diagnostic) using the following primers: 1S:  $5^{\prime}$ -ATTTTC- CAGGTSTGGCARAGGTC- $3^{\prime}$  (nucleotide 5476–5498 according  sequence) and 1AS:  $5^{\prime}$  $\mathrm{HIV-}2_{\mathrm{ROD}}$ -GCACATCCCCATGAAT- TTAG- $3^{\prime}$  (nucleotide 7920–7939). The RT step was carried out at  for 30 minutes, followed by  $94^{\circ}\mathrm{C}$  $50^{\circ}\mathrm{C}$  for 2 minutes. The PCR cycling parameters were 30 repeat cycles ( $(94^{\circ}\mathrm{C}$  for 30 seconds,  for 30 seconds, and  $68^{\circ}\mathrm{C}$  $55^{\circ}\mathrm{C}$  for 120 seconds) plus  $68^{\circ}\mathrm{C}$  for 7 minutes. This was followed by a second round of PCR using the following primers: 2S:  $5^{\prime}$ -AGARTCATGTGAYAARCAYTATTG-  (nucleotide 6779–6805) and 2R:  $5^{\prime}$  $\mathrm{GGA}{-}3^{\prime}$ -GCTGTTGCTGY- TGCTGCACTATCC- $3^{\prime}$  (nucleotide 7783–7806).

Nested-PCR reactions were performed with Taq polymerase (Applied Biosystems) and  $2.5~\upmu\mathrm{L}$  of internal 2S and 2R primers  $(10\,\mathrm{mmol/L})$ , following the manufacturer’s instructions. The PCR conditions consisted of initial denaturation at  $94^{\circ}\mathrm{C}$  for 5 minutes, followed by 40 repeated cycles ( $\mathrm{94^{\circ}C}$  for 30 seconds,  $60^{\circ}\mathrm{C}$  for 45 seconds,  $72^{\circ}\mathrm{C}$  for 90 seconds) plus  $72^{\circ}\mathrm{C}$  for 7 minutes. The HIV-2 V3-loop gene was sequenced using 2S and 2R primers, as inthenested-PCRreactions.Sequencingreactionswereperformed on an ABI Prism Dye Terminator kit using an automated se- quencer (ABI Prism 3130; Applied Biosystems). GenSearch soft- ware (Version 19; PhenoSystems) was used to edit and align the nucleotide sequences. When double chromatogram peaks were present, all amino acids possibilities were tested for tropism in- terpretation. HIV-2 V3-loop sequences were deposited in Gen- Bank with the following accession numbers: JN230718–JN230770.

Statistical Analyses Statistical analyses were performed with StatEL software (Version 2.5; AD Science). The association of CD4 cell count with viral tropism was tested by a Mann-Whitney test. The association of proportion of patients with detectable plasma HIV-2 RNA with viral tropism was tested by Fisher’s exact test. The association of each mutation in V3 loop with X4 or dual/mixed tropism was analyzed using the  $\chi^{2}$  test or Fisher’s exact test. Tropism pre- diction sensitivity was deﬁned as the proportion of phenotypically dual or X4 tropism HIV-2 viruses presenting with mutations of interest among all the isolates analyzed. Tropism prediction speciﬁcity was deﬁned as the proportion of phenotypically R5 tropism HIV-2 strains without any mutations of interest among all the isolates analyzed.

## RESULTS

Tropism Determination by Phenotypic Assay Results of the phenotypic tropism assay using GHOST(3) cell lines are depicted in Table 1. Cellular viability was  ${>}95\%$  in all samples tested. All controls of low-level infection showed con- cordant results, and cytopathogenic effects were observed in all samples displaying positive RTCN values, including the lowest ones (data not shown).

Of the 53 HIV-2 clinical isolates tested, 34 were classiﬁed as R5 tropic  $(64\%)$ , 12 as D/M tropic  $(23\%)$ , and 7 as X4 tropic  $(13\%)$ . Among the 7 X4 isolates, 4 exhibited a low-level signal on CCR5 cells. These isolates were considered X4 tropic, because their CCR5 RTCN values were  ${<}15\%$  of the corresponding CXCR4 RTCN values, as deﬁned in Materials and Methods. Among the  $34\,\mathrm{R}5$  isolates, 30 showed a positive signal exclusively on CCR5 cells, and 4 displayed a weak signal on CXCR4 cells. Thus, these 4 isolates were considered R5 tropic, because their CXCR4 RTCN values represented ${<}5\%$  of the corresponding CCR5 RTCN value.

Genotypic Tropism Assay: Analysis of V3-Loop Sequences Amino acid V3-loop sequences of the 53 HIV-2 clinical iso- lates are depicted in Table 1. D/M and X4 isolates were pooled. V3-loop sequences issued from R5-tropic isolates  $(\mathrm{~n~}=\,34)$ ) were then compared with the V3-loop sequences issued from D/M- or X4- (D/M-X4–) tropic isolates ( $\mathrm{{(n=19)}}$ ) to identify statistically signiﬁcant differences in residues between the 2 groups of sequences.

Eight positions in the V3-loop sequences were signiﬁcantly associated with D/M-X4 tropism in this series (Table 2). Any substitution at position 18 was associated with a D/M-X4 tro- pism in all cases ( $P<.00001)$ . The remaining mutations asso- ciated with D/M-X4 tropism were V19K/R  $(P~<~.00001)$ , S22 A/F/Y ( $P<.002)$ ), Q23R ( $P<.00001)$ , insertions at position 24  $(P<.00001)$ , I25L/Y ( $(P<.0004)$ , R28K ( $(P<.0004)$ , and R30K ( $P<.014\$ ). These mutations showed a speciﬁcity of  $100\%$  for D/M-X4 tropism, because they were never found in R5 iso- lates, except for the R28K and R30K mutations detected in 4 (speciﬁcity,  $88\%.$ ) and 5 (speciﬁcity,  $85\%$ ) R5 isolates, re- spectively. In addition, a V3-loop global net charge of more than  was also associated with D/M-X4 tropism ( $^{+6}$  $P<.00001)$ . Regarding the V3-loop sequence of the HIV-2 group A reference strain  $\scriptstyle(\mathrm{HIV-}2_{\mathrm{ROD}})$ , this exhibited 2 of the criteria previously as- sociated with D/M-X4 tropism, including a V3 global net charge of  $^{+7}$  and a L18H mutation. None of the genotypic determinants identiﬁed in our study seemed to be HIV-2 group dependent.



Thus, if we retained mutations with a sensitivity of  ${>}70\%$  and a speciﬁcity of  $100\%.$ , 4 major genotypic determinants of D/M-X4 tropism in the HIV-2 V3 loop can be deﬁned: (1) any mutation at residue 18 (sensitivity,  $100\%;$  speciﬁcity,  $100\%$ ); (2)  $\mathrm{V}19\,\mathrm{~K}/\mathrm{R}$  (sensitivity,  $89\%;$  speciﬁcity,  $100\%$ ; (3) V3-loop global-net charge (sensitivity,  $79\%$ ; speciﬁcity,  $100\%$ ), and (4) insertions at residue 24 (sensitivity,  $74\%;$  speciﬁcity,  $100\%.$ ). These major ge- notypic determinants were frequently found associated. Indeed, among the  $19\ \mathrm{D/M-X4}$  isolates, 4, 3, and 2 major genotypic de- terminants were present in  $10,\,6,$  and 3 isolates, respectively. The presence of 2 major genotypic determinants has a positive pre- dictive value of  $100\%$  ( $95\%$  conﬁdence interval,  $82\%{-}100\%)$  and a negative predictive value of  $100\%$  ( $95\%$  conﬁdence inter- val,  $90\%{-}100\%)$ ) in this series of HIV-2 clinical isolates.

Genotypic Tropism Assay: Analysis of V3-Loop Sequences Available in Literature The reanalysis of distinct data set sequences of HIV-2 V3-loop sequences currently available in the literature and in the Los Alamos National Laboratory (LANL) HIV database is reported in Table 3 [16, 17, 30, 31]. Concerning D/M-X4 HIV-2 isolates ( $\mathrm{~n~}=17^{\circ}$ ), of the major genotypic determinants of CXCR4 ${\geq}1$  coreceptor usage identiﬁed in our study was present in all X4 HIV-2 samples  $(\mathrm{n}\,=\,7)$ ) and in 4 of  $10~\mathrm{D/M}$  HIV-2 samples. Concerning the 33 R5 HIV-2 sequences available from other data sets, they showed that none exhibited any of the 4 major genotypic determinants identiﬁed in our study. Overall, we observed a good concordance in R5 and X4 tropism pre- dictions. The use of major genotypic determinants identiﬁed in our study shows a sensitivity of  $65\%$  and a speciﬁcity of  $100\%$  in detecting D/M-X4 viruses in these distinct data sets of HIV-2 sequences.



Association Between Tropism and Immunovirological Status Among the 34 patients infected by R5-tropic virus and the 19 patients infected by a D/M- or X4-tropic virus, CD4 cells counts were available at the time of collection in 25  $(74\%)$  and 16  $(84\%)$  of the patients, respectively. CD4 cells counts were signiﬁcantly different between these 2 groups ( $(P=.002)$ , with a median of  $286~\mathrm{cells}/\mathrm{mm}^{3}$  (interquartile range, 218–424) and 66 cells/mm3 (interquartile range, 29–159) in the R5- and D/M-X4–tropic groups, respectively (Table 4). Plasma HIV-2 RNA levels were available in  $29\,\mathrm{R}5$ -tropic patients  $(85\%)$  and in  $15~\mathrm{D/M}\cdot$ - or X4-tropic patients  $(79\%)$ . The proportion of pa- tients with detectable plasma HIV-2 RNA was higher in the D/M-X4–tropic group  $(\mathrm{n}\,=\,11;\,73\%)$ ) than in the R5-tropic group  $(\mathrm{n\;=\;14};\;48\%)$ ) (Table 4), but this difference was not statistically signiﬁcant ( $(P=.10)$ ).

## DISCUSSION

In the present study, which was carried out on 53 HIV-2 clinical isolates, we set up a phenotypic tropism assay with an automatic ﬁnal step based on cytometric analyses to assess infection level. One limitation of this phenotypic tropism test is the absence of parental GHOST cell lines that only express the CD4 receptor, which may provide more reliable determination of the threshold of infection but which are still unavailable at this time from the NIH AIDS Reagent Program. In our study, we identiﬁed 4 major V3-loop genotypic determinants of CXCR4 coreceptor use: mutations at residues 18 and 19, insertions at residue 24, and the V3-loop global net charge.

Previous studies on the V3-loop genotypic determinants of HIV-2 R5/X4 tropism have been based on smaller numbers of HIV-2 viruses, ranging from 5 to 18, with very few X4 and D/M viruses [15–17, 29–31]. Moreover, different methods of pheno- typic tropism assays were used in these studies, preventing a reli- able comparison of the results. Four of these studies showed an association between some V3-loop genotypic determinants and phenotypic tropism [15, 17, 29, 30]. No such association was found in the other 2 studies [16, 31]. In our study, the 4 major genotypic determinants of D/M-X4 tropism were all located in the gp105 V3-loop in its C-terminal part, which is already known to be a critical region for CCR5 or CXCR4 use in HIV-2 [30]. A V3-loop global net charge of more than  $^{+6}$  and the presence of mutations at residues 18 and 19 were previously observed in HIV-2 X4 or D/M viruses. However, the very low

Downloaded from  at University of Georgia on May 29, 2015

Abbreviations: D/M, dual/mixed; HIV-2, human immunodeﬁciency virus type 2; IQR, interquartile range.

a Viremic patients were deﬁned as those with plasma HIV-2 RNA levels $>\!100$  copies/mL.

b D/M-X4–tropic group represents pooled data for the D/M- and  $\mathsf{X4}$ -tropic groups.

number of D/M-X4 viruses in these studies prevents statistical analyses [15, 17, 30].

Regarding HIV-1 infection, the gp120 V3-loop major geno- typic determinants of CXCR4 coreceptor use are at positions 11 and 25 and the V3-loop global net charge; the latter is also described in HIV-2 infection. Conversely, positions 11 and 25 were not found associated with HIV-2 tropism prediction. However, the large genotypic differences between HIV-1 and HIV-2 V3 loops render it difﬁcult to make such comparisons.

Regarding viral tropism analysis of HIV-2 gp105 V3-loop se- quences with a known coreceptor usage available in the literature [16, 17, 30, 31], a sensitivity of  $65\%$  and a speciﬁcity of  $100\%$  in detecting D/M-X4 viruses was shown using the major genotypic determinants of CXCR4 usage we identiﬁed in our study. In all HIV-2 clinical isolates from our series, a mutation at residue 18 is sufﬁcient to predict D/M-X4 tropism. However, among the 7 X4 isolates from the previous study data set, 6 displayed a mutant residue at residue 18; the remaining 1 exhibited only a global net charge of  $^{+7}$  [16]. This suggests that residue 18 alone is probably not sufﬁcient to predict CXCR4 coreceptor usage. In this rean- alysis, the lower concordance was observed in samples exhibiting a D/M-tropic mixture, because 6 of the 10 reported D/M- tropic viruses were classiﬁed as R5 by analysis with the 4 major genotypic determinants. This observation may be due to the various phe- notypic tropism assays used in these studies and to the possible differences in double chromatogram peaks interpretation. Similar rates of sensitivity and speciﬁcity were obtained for tropism ge- notypic prediction in HIV-1 infection. Thus, in the ANRS Geno- TropismStudy,which evaluatedthe genotypicpredictionof HIV-1 coreceptor use versus the phenotypic Troﬁle assay, the highest sensitivity of genotypic-tropism prediction was  ${\sim}60\%$  according to different genotypic algorithms [38]. For all algorithms tested, the speciﬁcity to detect X4 compared with the Troﬁle test ranged from  $79.3\%$  to  $98.5\%$  [38]. Similar results of genotypic prediction compared to the standard Troﬁle assay have been reported after analysis of the MOTIVATE-1 study database [39].



In conclusion, in our study we established, for the ﬁrst time, a strong association between HIV-2 phenotypic tropism and V3-loop genotypic determinants based on a large series of pa- tients offering the opportunity to identify R5- and/or X4-tropic viruses. This genotypic tropism test could help improve the knowledge on HIV-2 pathogenesis by describing the evolution of tropism during the natural history of infection and also by describing the tropism in different compartments.

### Notes

Acknowledgments.C. C., F. D., S. M., F. B. V., and D. D. contributed to the study concept. B. V., A. S., and G. C. performed genotypic tests. B. V., M. H. N., A. S., and G. C. performed phenotypic tests. C. C., L. L., F. D., S. M., and D. D. contributed to the analysis and interpretation of the data. C. C., B. V., F. D., F. B. V., and D. D. contributed to the writing of the manuscript. All authors contributed to the critical reviewing of the manuscript.

Financial support.This work was supported by the French National Agency for Research on AIDS and Viral Hepatitis (ANRS) and by the European Community’s Seventh Framework Programme (grant FP7/ 2007–2013) under the project Collaborative HIV and Anti-HIV Drug Resistance Network (CHAIN) (grant 223131).

Potential conﬂicts of interest.All authors: No reported conﬂicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conﬂicts of Interest. Conﬂicts that the editors consider relevant to the content of the manuscript have been disclosed.

### References

1. Valadas E, Francxa L, Sousa S, Antunes F. 20 years of HIV-2 infection in Portugal: trends and changes in epidemiology. Clin Infect Dis 2009; 48:1166–7.

2. Brunet S, Thierry D, Barin F, et al. Surveillance de l’infection a\` VIH-sida en France, 2007. BEH 2008; 45:434–42.

3. Roquebert B, Damond F, Collin G, et al. HIV-2 integrase gene poly- morphism and phenotypic susceptibility of HIV-2 clinical isolates to the integrase inhibitors raltegravir and elvitegravir in vitro. J Antimicrob Chemother 2008; 62:914–20.

4. Ntemgwa ML, d’Aquin Toni T, Brenner BG, Camacho RJ, Wainberg MA. Antiretroviral drug resistance in human immunodeﬁciency virus type 2. Antimicrob Agents Chemother 2009; 53:3611–9.

5. Ren J, Bird LE, Chamberlain PP, Stewart-Jones GB, Stuart DI, Stammers DK. Structure of HIV-2 reverse transcriptase at 2.35-A resolution and the mechanism of resistance to non-nucleoside inhibitors. Proc Natl Acad Sci U.S.A 2002; 99:14410–5.

6. Witvrouw M, Pannecouque C, Switzer WM, Folks TM, De Clercq E, Heneine W. Susceptibility of HIV-2, SIV and SHIV to various anti-HIV-1 compounds: implications for treatment and postexposure prophylaxis. Antivir Ther 2004; 9:57–65.

7. Poveda E, Briz V, Soriano V. Enfuvirtide, the ﬁrst fusion inhibitor to treat HIV infection. AIDS Rev 2005; 7:139–47.

8. Desbois D, Roquebert B, Peytavin G, et al. In vitro phenotypic susceptibility of human immunodeﬁciency virus type 2 clinical isolates to protease inhibitors. Antimicrob Agents Chemother 2008; 52:1545–8.

9. Willey S, Peters PJ, Sullivan WM, Dorr P, Perros M, Clapham PR. Inhibition of CCR5-mediated infection by diverse R5 and R5X4 HIV and SIV isolates using novel small molecule inhibitors of CCR5: effects of viral diversity, target cell and receptor density. Antiviral Res 2005; 68:96–108.

10. Armstrong-James D, Stebbing J, Scourﬁeld A, et al. Clinical outcome in resistant HIV-2 infection treated with raltegravir and maraviroc. Antiviral Res 2010; 86:224–6.

11. Stegmann S, Manea ME, Charpentier C, et al. Foscarnet as salvage therapy in HIV-2-infected patient with antiretroviral treatment failure. J Clin Virol 2010; 47:79–81.

12. McKnight A, Dittmar MT, Moniz-Periera J, et al. A broad range of chemokine receptors are used by primary isolates of human immu- nodeﬁciency virus type 2 as coreceptors with CD4. J Virol 1998; 72:4065–71.

13. Mo¨rner A, Bjo¨rndal A, Leandersson A, Albert J, Bjo¨rling E, Jansson M. CCR5 or CXCR4 is required for efﬁcient infection of peripheral blood mononuclear cells by promiscuous human immunodeﬁciency virus type 2 primary isolates. AIDS Res Hum Retroviruses 2002; 18:193–200. 14. Guillon C, van der Ende ME, Boers PH, Gruters RA, Schutten M, Osterhaus AD. Coreceptor usage of human immunodeﬁciency virus type 2 primary isolates and biological clones is broad and does not correlate with their syncytium-inducing capacities. J Virol 1998; 72:6260–3.

15. Mo¨rner A, Bjo¨rndal A, Albert J, et al. Primary human immunodeﬁ- ciency virus type 2 (HIV-2) isolates, like HIV-1 isolates, frequently use CCR5 but show promiscuity in coreceptor usage. J Virol 1999; 73:2343–9.

16. Owen SM, Ellenberger D, Rayﬁeld M, et al. Genetically divergent strains of human immunodeﬁciency virus type 2 use multiple coreceptors for viral entry. J Virol 1998; 72:5425–32.

17. Shi Y, Brandin E, Vincic E, et al. Evolution of human immunodeﬁciency virus type 2 coreceptor usage, autologous neutralization, envelope se- quence and glycosylation. J Gen Virol 2005; 86:3385–96.

18. Blaak H, Boers PH, Gruters RA, Schuitemaker H, van der Ende ME, Osterhaus AD. CCR5, GPR15, and CXCR6 are major coreceptors of human immunodeﬁciency virus type 2 variants isolated from individuals with and without plasma viremia. J Virol 2005; 79:1686–700.

19. Zhang Y, Lou B, Lal RB, Gettie A, Marx PA, Moore JP. Use of inhibitors to evaluate coreceptor usage by simian and simian/human immunode- ﬁciency viruses and human immunodeﬁciency virus type 2 in primary cells. J Virol 2000; 74:6893–10.

20. Kuhmann SE, Hartley O. Targeting chemokine receptors in HIV: a status report. Annu Rev Pharmacol Toxicol 2008; 48:425–61.

21. Tsibris AM, Kuritzkes DR. Chemokine antagonists as therapeutics: focus on HIV-1. Annu Rev Med 2007; 58:445–59.

22. Whitcomb JM, Huang W, Fransen S, et al. Development and charac- terization of a novel single-cycle recombinant-virus assay to determine human immunodeﬁciency virus type 1 coreceptor tropism. Antimicrob Agents Chemother 2007; 51:566–75.

23. Jensen MA, van ’t Wout AB. Predicting HIV-1 coreceptor usage with sequence analysis. AIDS Rev 2003; 5:104–12.

24. Beerenwinkel N, Sing T, Lengauer T, et al. Computational methods for the design of effective therapies against drug resistant HIV strains. Bioinformatics 2005; 21:3943–50.

25. Web PSSM. http://indra.mullins.microbiol.washington.edu/webpssm/ index.html. Accessed 10 March 2011.

26. Geno2pheno coreceptor. http://coreceptor.bioinf.mpi-inf.mpg.de/ index.php. Accessed 10 March 2011.

27. Skrabal K, Low AJ, Dong W, et al. Determining human immunodeﬁ- ciency virus coreceptor use in a clinical setting: degree of correlation between two phenotypic assays and a bioinformatic model. J Clin Microbiol 2007; 45:279–84.

28. Raymond S, Delobel P, Mavigner M, et al. Correlation between geno- typic predictions based on V3 sequences and phenotypic determination of HIV-1 tropism. AIDS 2008; 22:F11–6.

29. Albert J, Sta˚lhandske P, Marquina S, et al. Biological phenotype of HIV type 2 isolates correlates with V3 genotype. AIDS Res Hum Retroviruses 1996; 12:821–8.

30. Isaka Y, Sato A, Miki S, et al. Small amino acid changes in the V3 loop of human immunodeﬁciency virus type 2 determines the coreceptor usage for CXCR4 and CCR5. Virology 1999; 264:237–43.

31. Kulkarni S, Tripathy S, Agnihotri K, et al. Indian primary HIV-2 isolates and relationship between V3 genotype, biological phenotype and cor- eceptor usage. Virology 2005; 337:68–75.

32. Davis HE, Rosinski M, Morgan JR, Yarmush ML. Charged polymers modulate retrovirus transduction via membrane charge neutralization and virus aggregation. Biophys J 2004; 86:1234–42.

33. Ho WZ, Cherukuri R, Ge SD, et al. Centrifugal enhancement of human immunodeﬁciency virus type 1 infection and human cytomegalovirus gene expression in human primary monocyte/macrophages in vitro. J Leukoc Biol 1993; 53:208–12.

34. Pietroboni GR, Harnett GB, Bucens MR. Centrifugal enhancement of human immunodeﬁciency virus (HIV) and human herpesvirus type 6 (HHV-6) infection in vitro. J Virol Methods 1989; 24:85–90.

35. Vo¨dro¨s D, Tscherning-Casper C, Navea L, Schols D, De Clercq E, Fenyo¨ EM. Quantitative evaluation of HIV-1 coreceptor use in the GHOST3 cell assay. Virology 2001; 291:1–11.

36. Cho MW, Lee MK, Carney MC, Berson JF, Doms RW, Martin MA. Identiﬁcation of determinants on a dualtropic human immunodeﬁ- ciency virus type 1 envelope glycoprotein that confer usage of CXCR4. J Virol 1998; 72:2509–15.

37. Schols D, Este´ JA, Cabrera C, De Clercq E. T-cell-line-tropic human immunodeﬁciency virus type 1 that is made resistant to stromal cell- derived factor 1alpha contains mutations in the envelope gp120 but does not show a switch in coreceptor use. J Virol 1998; 72:4032–7.

38. Recordon-Pinson P, Soulie´ C, Flandre P, et al. Evaluation of the genotypic prediction of HIV-1 coreceptor use versus a phenotypic assay and correlation with the virological response to maraviroc: the ANRS GenoTropism study. Antimicrob Agents Chemother 2010; 54:3335–40.

39. McGovern RA, Thielen A, Mo T, et al. Population-based V3 genotypic tropism assay: a retrospective analysis using screening samples from the A4001029 and MOTIVATE studies. AIDS 2010; 24:2517–25.
Molecular Determinants of HIV-2 R5–X4 Tropism in the V3 Loop: Development of a New Genotypic Tool

### Benoit Visseaux, Margarita Hurtado-Nedelec, Charlotte Charpentier,1,4 Gilles Collin,1,4 Alexandre Storto,1,4

Sophie Matheron, Lucile Larrouy, Florence Damond, Francxoise Brun-Ve´zinet, Diane Descamps, and the ANRS CO 05 HIV-2 Cohort Laboratoire de Virologie,De´partement d'He´matologie et Immunologie, and 3Service des Maladies Infectieuses et Tropicales, Assistance Publique-Hoˆpitaux de Paris, Hoˆpital Bichat-Claude Bernard, and $^{4}\mathsf{E A}$  4409 Paris-Diderot University, France



Objective.The use of CCR5 inhibitors requires a tool to predict human immunodeﬁciency virus type 2 (HIV-2) tropism, as established in HIV-1. The aim of our study was to identify genotypic determinants of HIV-2 tropism located in the gp105 V3 loop.

Methods.HIV-2 tropism phenotypic assays were performed on 53 HIV-2 clinical isolates using GFP expressing human osteosarcoma T4 [GHOST(3)] cell lines expressing CD4 and CCR5 or CXCR4 coreceptors. The gp105 V3 loop was sequenced and analyzed.

Results.Thirty-four HIV-2 isolates were classiﬁed as R5, 7 as X4, and 12 as X4/R5 (dual). Substitution at residue 18 was always associated with a dual/X4 tropism ( $P<.00001)$ . The following determinants were associated with dual/X4 tropism: a global net charge of more than  $^{+6}$   $(P<.00001)$ , V19K/R mutation ( $\left(P<.00001\right)$ ), S22A/F/Y mutation ( $(P<.002)$ ), Q23R mutation ( $P<.00001)$ , and insertions at residue 24 ( $P<.00001)$ ), I25L/Y ( $P<.0004)$ ), R28K ( $P<.0004)$ , and R30K ( $P<.014)$ ). These mutations were not found in R5 isolates, except R28K and R30K, which were detected in 4 and 5 R5 isolates, respectively. The 4 major genotypic determinants of dual/X4 tropism were mutation at residue 18,  $\mathrm{V}19~\mathrm{K}/\mathrm{R}$  mutation, insertions at residue 24, and V3 global net charge.

Conclusions.We established a strong association between HIV-2 phenotypic tropism and V3-loop sequences, allowing for the prediction of R5- and/or X4-tropic viruses in HIV-2 infection.

The human immunodeﬁciency virus (HIV) is character- ized by a high level of genetic diversity within HIV type 1 (HIV-1) and HIV type 2 (HIV-2). HIV-2 infections are mainly restricted to West Africa, including Guinea-Bissau, Gambia, Senegal, and Guinea. Some European countries are also concerned about HIV-2 infection, which repre- sents  $5\%$  of HIV infection in a series of patients in Por- tugal [1] and  $2\%$  of the new HIV infections in France [2].

HIV-2 clinical isolates appear to be susceptible to nu- cleoside reverse-transcriptase inhibitors and to the new drug class of integrase inhibitors [3, 4]. However, they are naturally resistant to nonnucleoside reverse-transcriptase inhibitors [5, 6] and to fusion-inhibitor drug classes [6, 7]. In addition, HIV-2 shows decreased susceptibility to some protease inhibitors, such as amprenavir, atazanavir, and tipranavir [6, 8]. In this context, the new class of CCR5 coreceptor (R5) inhibitors with maraviroc may provide a new therapeutic opportunity in HIV-2 infection man- agement. However, few data are available on maraviroc use in HIV-2 infection. One study found in vitro antiviral activity of nonlicensed CCR5 inhibitors on HIV-2 [9] and 2 HIV-2–infected patients with a multiresistant virus showed an early virological response after receiving a multiple-drug salvage regimen with maraviroc [10, 11]. The process of HIV-2 entry involves identical steps to HIV-1, including similar virus-cell interactions, ﬁrst be- tween the HIV-2 gp105 envelope glycoprotein and CD4 receptor and then between the gp105 V3 loop and CCR5 or CXCR4 coreceptors [12, 13]. Furthermore, HIV-2 seems to be able to use in vitro a broad range of coreceptors, such as CCR1, CCR2b, CCR3, CCR8, BOB/GPR15, or BONZO/CXCR6 [12, 14–17]. However, discrepancies exist between studies and cell lines models for tropism determination. Indeed, U87 cell lines showed higher level of nonspeciﬁc infection than GFP expressing human osteosarcoma T4 [GHOST(3)] cell lines [17, 18]. In the GHOST(3) cell line model, only CCR5, GPR15, and CXCR6 were efﬁciently used in vitro by R5-tropic HIV-2 viruses, and only CXCR4 was efﬁciently used by X4-tropic HIV-2 virus [18]. Moreover, 2 studies assessed in vitro the relative contribution of these different coreceptors in HIV-2 infection using coreceptor- speciﬁcinhibitorsonperipheral-bloodmononuclearcells (PBMCs) or a PBMC model with  $\mathrm{CCR5^{+/+}}$ and  $\mathrm{CCR5^{-/-}}$  blood donors, showing that only CCR5- or CXCR4-dependent infection is really efﬁcient on PBMC [13, 19].



CCR5 inhibitors are ineffective against the CXCR4 (X4)–using virus [20, 21]. Thus, it is mandatory to assess viral tropism in order to determine a patient’s eligibility for maraviroc therapy. In HIV-1 infection, tropism can be determined by a phenotypic recombi- nant test (Troﬁle; Monogram Biosciences) [22] or by a genotypic test that analyzes the gp120 V3-loop sequences [23, 24]. Genotypic interpretation of V3-loop sequences for tropism determination can be performed using different algorithms [25, 26] that take into account the major genotypic determinants of coreceptors used in the HIV-1 genetic context [27, 28].

In HIV-2 infection, several studies have evaluated the corre- lations between the gp105 V3-loop sequence and HIV-2 tropism, as assessed by phenotypic tests [15–17, 29–31]. All these studies have reported a few HIV-2 strains, and have reached discordant conclusions. The aim of this study was to establish a genotypic tool for HIV-2-tropism determination, as a ﬁrst step in studying tropism implications in HIV-2 infection.

## MATERIALS AND METHODS

Study Patients and Virus Stocks Fifty-three HIV-2 clinical isolates, from 53 HIV-2–infected, maraviroc-naive patients of the French Agence Nationale de Re- cherches sur le SIDA et les he´patites virales (ANRS) HIV-2 cohort  $(\mathrm{CO}~05)$ , were isolated by cocultivation of phytohemagglutinin- activated PBMCs from a pool of healthy blood donors, as de- scribed elsewhere [3]. Written consent was provided by all patients at the time of inclusion in the French ANRS CO 05 HIV-2 cohort. Among the 53 patients, 37  $(70\%)$  were infected with HIV-2 group A, 15( $(28\%)$  with HIV-2 groupB, and1 with HIV-2groupH  $(2\%)$ . The HIV-2 group A reference strain,  $\mathrm{HV-}2_{\mathrm{ROD}},$  using both CCR5 and CXCR4 coreceptors, was included in our series.

Cell Lines Peripheral-blood mononuclear cells were obtained from HIV- seronegativeblooddonors.Theywerestimulatedwith phytohemagglutinin and cultured, as described elsewhere [3]. GHOST(3) cell lines are human osteosarcoma cells engineered to stably express the CD4 receptor and one HIV coreceptor, CCR5 or CXCR4. Both GHOST(3) cell lines (CD4 plus CCR5 or CD4 plus CXCR4) also carry a humanized green ﬂuorescent protein (GFP) gene under the control of a HIV-2 long terminal repeat [15]. Infection with HIV and consecutive production of the Tat protein induces expression of GFP, which can be detected by ﬂuorescent microscopy or cytometric analysis. GHOST(3) cell lines were cultured in Dulbecco’s Modiﬁed Eagle Medium (DMEM) medium (Invitrogen) containing  $10\%$  fetal bovine , $(100~\upmu\mathrm{g/mL})$  $(500~\upmu\mathrm{g/mL})$  and puromycin  $(1~\upmu\mathrm{g/mL})$ ). GHOST(3) High CCR5 coreceptors (Hi-5) cells and GHOST(3) CXCR4 cells were provided by the National Institutes of Health (NIH) AIDS Reagent Program. At the time of this work, parental cell lines expressing only the CD4 receptor without any coreceptor were not available and were still not available at the writing of this work.



Phenotypic Tropism Assay GHOST(3) cells were seeded in 24-well plates at  $40\;000$  per well in growth medium and incubated for 24 hours at  $37^{\circ}\mathrm{C}$  in  $5\%$  carbon dioxide  $(\mathrm{CO}_{2})$ . After incubation, the culture medium was removed from the wells, and  $300~\upmu\mathrm{L}$  of cell-free virus, at a titer between  $10^{7}$  and  $10^{8}\;\mathrm{copies/mL}$  $10^{8}$  and contained in DMEM, with  $10\%$  FBS and  polybrene, was added to the cells. Plates were immediately $7.5\,\upmu\mathrm{g/L}$  centrifuged for 30 minutes at  $2300\ g.$ . The centrifugation step and the addition of polybrene, an anionic polymer that enhances virus- cell contacts by shielding charge and virus-aggregation mecha- nisms, were added to increase infection efﬁciency [32–34]. Plates were incubated for 4 hours at  $37^{\circ}\mathrm{C}$  in  $5\%\ \mathrm{CO}_{2}$ . The viral super- natant was then removed, and  $1\;\mathrm{mL}$  of culture medium was added to each well. Plates were incubated for 3 days at  $37^{\circ}\mathrm{C}$  in  $5\%\;\mathrm{CO}_{2}$ . Cells were then observed for cytopathogenic effects, the culture  of trypsin was added. After $90~\upmu\mathrm{L}$  5 minutes of incubation at  $37^{\circ}\mathrm{C},$  trypsin was inactivated by the addition of  $300~\upmu\mathrm{L}$  of phosphate-buffered saline and  $20\%$  FBS. Cells were analyzed within an hour by ﬂow cytometry, after staining with propidium iodide in order to assess cellular viability. A negative control, containing only culture medium, was included in each experiment. The ratio-to-cell-negative (RTCN) value was used to quantify cell infection [18, 35]. The RTCN value is the ratio of the percentage of ﬂuorescence-positive cells to their mean ﬂuorescence intensity (FI), calculated as follows:  $\mathrm{RTCN}=(\%$  of GFP-positive cells) of infected cells/( $\%$  of GFP-positive cells $\times\mathrm{{FI}}_{\ell}$   of uninfected cells. A RTCN value of 5 was taken as the $\times\mathrm{\FI})$  negative cutoff threshold. In our series, when the proportion of GFP-positive cells was  ${<}10\%$ , the phenotypic tropism assay was performed again using a 2-fold larger viral inoculum. No dis- cordance was found with these controls (data not shown).



GHOST(3) cell lines are known to endogenously express in low-level proportion the CXCR4 coreceptor [36, 37]. Thus, low levels of infection were described in previous study with HIV-2 X4 isolates in GHOST(3) cell lines [18]. In regard to these latter data, in our study all HIV-2 isolates with a CCR5 cell RTCN value  of the CXCR4 cell RTCN value were considered X4 tropic. ${<}15\%$  A very low-level infection of parental GHOST cell lines has been described elsewhere in some HIV-2 isolates, independently of CCR5 or CXCR4 coreceptors [18]. The proposed mechanism could be coreceptor-independent infection or alternative cor- eceptor usage. However, these infections are infrequent and have a very low efﬁciency. In regards to the data obtained by Blaak et al [18], all HIV-2 isolates that have a CXCR4 RTCN value  ${<}5\%$  of the CCR5 cell RTCN value were considered R5 tropic.



HIV-2 Gene Sequencing HIV-2 RNA was extracted from 1 mL of viral supernatant using a Total NA Large Volume kit (Roche Diagnostic) on a MagnaPure automate (Roche Diagnostic), according to the manufacturer’s recommendations. A 1000–base pair fragment of gp105, encom- passing the V3 loop, was ﬁrst ampliﬁed by reverse-transcription polymerase chain reaction (RT-PCR) according to the manu- facturer’s instructions (Titan One-Tube RT-PCR kit; Roche Diagnostic) using the following primers: 1S:  $5^{\prime}$ -ATTTTC- CAGGTSTGGCARAGGTC- $3^{\prime}$  (nucleotide 5476–5498 according  sequence) and 1AS:  $5^{\prime}$  $\mathrm{HIV-}2_{\mathrm{ROD}}$ -GCACATCCCCATGAAT- TTAG- $3^{\prime}$  (nucleotide 7920–7939). The RT step was carried out at  for 30 minutes, followed by  $94^{\circ}\mathrm{C}$  $50^{\circ}\mathrm{C}$  for 2 minutes. The PCR cycling parameters were 30 repeat cycles ( $(94^{\circ}\mathrm{C}$  for 30 seconds,  for 30 seconds, and  $68^{\circ}\mathrm{C}$  $55^{\circ}\mathrm{C}$  for 120 seconds) plus  $68^{\circ}\mathrm{C}$  for 7 minutes. This was followed by a second round of PCR using the following primers: 2S:  $5^{\prime}$ -AGARTCATGTGAYAARCAYTATTG-  (nucleotide 6779–6805) and 2R:  $5^{\prime}$  $\mathrm{GGA}{-}3^{\prime}$ -GCTGTTGCTGY- TGCTGCACTATCC- $3^{\prime}$  (nucleotide 7783–7806).

Nested-PCR reactions were performed with Taq polymerase (Applied Biosystems) and  $2.5~\upmu\mathrm{L}$  of internal 2S and 2R primers  $(10\,\mathrm{mmol/L})$ , following the manufacturer’s instructions. The PCR conditions consisted of initial denaturation at  $94^{\circ}\mathrm{C}$  for 5 minutes, followed by 40 repeated cycles ( $\mathrm{94^{\circ}C}$  for 30 seconds,  $60^{\circ}\mathrm{C}$  for 45 seconds,  $72^{\circ}\mathrm{C}$  for 90 seconds) plus  $72^{\circ}\mathrm{C}$  for 7 minutes. The HIV-2 V3-loop gene was sequenced using 2S and 2R primers, as inthenested-PCRreactions.Sequencingreactionswereperformed on an ABI Prism Dye Terminator kit using an automated se- quencer (ABI Prism 3130; Applied Biosystems). GenSearch soft- ware (Version 19; PhenoSystems) was used to edit and align the nucleotide sequences. When double chromatogram peaks were present, all amino acids possibilities were tested for tropism in- terpretation. HIV-2 V3-loop sequences were deposited in Gen- Bank with the following accession numbers: JN230718–JN230770.

Statistical Analyses Statistical analyses were performed with StatEL software (Version 2.5; AD Science). The association of CD4 cell count with viral tropism was tested by a Mann-Whitney test. The association of proportion of patients with detectable plasma HIV-2 RNA with viral tropism was tested by Fisher’s exact test. The association of each mutation in V3 loop with X4 or dual/mixed tropism was analyzed using the  $\chi^{2}$  test or Fisher’s exact test. Tropism pre- diction sensitivity was deﬁned as the proportion of phenotypically dual or X4 tropism HIV-2 viruses presenting with mutations of interest among all the isolates analyzed. Tropism prediction speciﬁcity was deﬁned as the proportion of phenotypically R5 tropism HIV-2 strains without any mutations of interest among all the isolates analyzed.

## RESULTS

Tropism Determination by Phenotypic Assay Results of the phenotypic tropism assay using GHOST(3) cell lines are depicted in Table 1. Cellular viability was  ${>}95\%$  in all samples tested. All controls of low-level infection showed con- cordant results, and cytopathogenic effects were observed in all samples displaying positive RTCN values, including the lowest ones (data not shown).

Of the 53 HIV-2 clinical isolates tested, 34 were classiﬁed as R5 tropic  $(64\%)$ , 12 as D/M tropic  $(23\%)$ , and 7 as X4 tropic  $(13\%)$ . Among the 7 X4 isolates, 4 exhibited a low-level signal on CCR5 cells. These isolates were considered X4 tropic, because their CCR5 RTCN values were  ${<}15\%$  of the corresponding CXCR4 RTCN values, as deﬁned in Materials and Methods. Among the  $34\,\mathrm{R}5$  isolates, 30 showed a positive signal exclusively on CCR5 cells, and 4 displayed a weak signal on CXCR4 cells. Thus, these 4 isolates were considered R5 tropic, because their CXCR4 RTCN values represented ${<}5\%$  of the corresponding CCR5 RTCN value.

Genotypic Tropism Assay: Analysis of V3-Loop Sequences Amino acid V3-loop sequences of the 53 HIV-2 clinical iso- lates are depicted in Table 1. D/M and X4 isolates were pooled. V3-loop sequences issued from R5-tropic isolates  $(\mathrm{~n~}=\,34)$ ) were then compared with the V3-loop sequences issued from D/M- or X4- (D/M-X4–) tropic isolates ( $\mathrm{{(n=19)}}$ ) to identify statistically signiﬁcant differences in residues between the 2 groups of sequences.

Eight positions in the V3-loop sequences were signiﬁcantly associated with D/M-X4 tropism in this series (Table 2). Any substitution at position 18 was associated with a D/M-X4 tro- pism in all cases ( $P<.00001)$ . The remaining mutations asso- ciated with D/M-X4 tropism were V19K/R  $(P~<~.00001)$ , S22 A/F/Y ( $P<.002)$ ), Q23R ( $P<.00001)$ , insertions at position 24  $(P<.00001)$ , I25L/Y ( $(P<.0004)$ , R28K ( $(P<.0004)$ , and R30K ( $P<.014\$ ). These mutations showed a speciﬁcity of  $100\%$  for D/M-X4 tropism, because they were never found in R5 iso- lates, except for the R28K and R30K mutations detected in 4 (speciﬁcity,  $88\%.$ ) and 5 (speciﬁcity,  $85\%$ ) R5 isolates, re- spectively. In addition, a V3-loop global net charge of more than  was also associated with D/M-X4 tropism ( $^{+6}$  $P<.00001)$ . Regarding the V3-loop sequence of the HIV-2 group A reference strain  $\scriptstyle(\mathrm{HIV-}2_{\mathrm{ROD}})$ , this exhibited 2 of the criteria previously as- sociated with D/M-X4 tropism, including a V3 global net charge of  $^{+7}$  and a L18H mutation. None of the genotypic determinants identiﬁed in our study seemed to be HIV-2 group dependent.



Thus, if we retained mutations with a sensitivity of  ${>}70\%$  and a speciﬁcity of  $100\%.$ , 4 major genotypic determinants of D/M-X4 tropism in the HIV-2 V3 loop can be deﬁned: (1) any mutation at residue 18 (sensitivity,  $100\%;$  speciﬁcity,  $100\%$ ); (2)  $\mathrm{V}19\,\mathrm{~K}/\mathrm{R}$  (sensitivity,  $89\%;$  speciﬁcity,  $100\%$ ; (3) V3-loop global-net charge (sensitivity,  $79\%$ ; speciﬁcity,  $100\%$ ), and (4) insertions at residue 24 (sensitivity,  $74\%;$  speciﬁcity,  $100\%.$ ). These major ge- notypic determinants were frequently found associated. Indeed, among the  $19\ \mathrm{D/M-X4}$  isolates, 4, 3, and 2 major genotypic de- terminants were present in  $10,\,6,$  and 3 isolates, respectively. The presence of 2 major genotypic determinants has a positive pre- dictive value of  $100\%$  ( $95\%$  conﬁdence interval,  $82\%{-}100\%)$  and a negative predictive value of  $100\%$  ( $95\%$  conﬁdence inter- val,  $90\%{-}100\%)$ ) in this series of HIV-2 clinical isolates.

Genotypic Tropism Assay: Analysis of V3-Loop Sequences Available in Literature The reanalysis of distinct data set sequences of HIV-2 V3-loop sequences currently available in the literature and in the Los Alamos National Laboratory (LANL) HIV database is reported in Table 3 [16, 17, 30, 31]. Concerning D/M-X4 HIV-2 isolates ( $\mathrm{~n~}=17^{\circ}$ ), of the major genotypic determinants of CXCR4 ${\geq}1$  coreceptor usage identiﬁed in our study was present in all X4 HIV-2 samples  $(\mathrm{n}\,=\,7)$ ) and in 4 of  $10~\mathrm{D/M}$  HIV-2 samples. Concerning the 33 R5 HIV-2 sequences available from other data sets, they showed that none exhibited any of the 4 major genotypic determinants identiﬁed in our study. Overall, we observed a good concordance in R5 and X4 tropism pre- dictions. The use of major genotypic determinants identiﬁed in our study shows a sensitivity of  $65\%$  and a speciﬁcity of  $100\%$  in detecting D/M-X4 viruses in these distinct data sets of HIV-2 sequences.



Association Between Tropism and Immunovirological Status Among the 34 patients infected by R5-tropic virus and the 19 patients infected by a D/M- or X4-tropic virus, CD4 cells counts were available at the time of collection in 25  $(74\%)$  and 16  $(84\%)$  of the patients, respectively. CD4 cells counts were signiﬁcantly different between these 2 groups ( $(P=.002)$ , with a median of  $286~\mathrm{cells}/\mathrm{mm}^{3}$  (interquartile range, 218–424) and 66 cells/mm3 (interquartile range, 29–159) in the R5- and D/M-X4–tropic groups, respectively (Table 4). Plasma HIV-2 RNA levels were available in  $29\,\mathrm{R}5$ -tropic patients  $(85\%)$  and in  $15~\mathrm{D/M}\cdot$ - or X4-tropic patients  $(79\%)$ . The proportion of pa- tients with detectable plasma HIV-2 RNA was higher in the D/M-X4–tropic group  $(\mathrm{n}\,=\,11;\,73\%)$ ) than in the R5-tropic group  $(\mathrm{n\;=\;14};\;48\%)$ ) (Table 4), but this difference was not statistically signiﬁcant ( $(P=.10)$ ).

## DISCUSSION

In the present study, which was carried out on 53 HIV-2 clinical isolates, we set up a phenotypic tropism assay with an automatic ﬁnal step based on cytometric analyses to assess infection level. One limitation of this phenotypic tropism test is the absence of parental GHOST cell lines that only express the CD4 receptor, which may provide more reliable determination of the threshold of infection but which are still unavailable at this time from the NIH AIDS Reagent Program. In our study, we identiﬁed 4 major V3-loop genotypic determinants of CXCR4 coreceptor use: mutations at residues 18 and 19, insertions at residue 24, and the V3-loop global net charge.

Previous studies on the V3-loop genotypic determinants of HIV-2 R5/X4 tropism have been based on smaller numbers of HIV-2 viruses, ranging from 5 to 18, with very few X4 and D/M viruses [15–17, 29–31]. Moreover, different methods of pheno- typic tropism assays were used in these studies, preventing a reli- able comparison of the results. Four of these studies showed an association between some V3-loop genotypic determinants and phenotypic tropism [15, 17, 29, 30]. No such association was found in the other 2 studies [16, 31]. In our study, the 4 major genotypic determinants of D/M-X4 tropism were all located in the gp105 V3-loop in its C-terminal part, which is already known to be a critical region for CCR5 or CXCR4 use in HIV-2 [30]. A V3-loop global net charge of more than  $^{+6}$  and the presence of mutations at residues 18 and 19 were previously observed in HIV-2 X4 or D/M viruses. However, the very low

Downloaded from  at University of Georgia on May 29, 2015

Abbreviations: D/M, dual/mixed; HIV-2, human immunodeﬁciency virus type 2; IQR, interquartile range.

a Viremic patients were deﬁned as those with plasma HIV-2 RNA levels $>\!100$  copies/mL.

b D/M-X4–tropic group represents pooled data for the D/M- and  $\mathsf{X4}$ -tropic groups.

number of D/M-X4 viruses in these studies prevents statistical analyses [15, 17, 30].

Regarding HIV-1 infection, the gp120 V3-loop major geno- typic determinants of CXCR4 coreceptor use are at positions 11 and 25 and the V3-loop global net charge; the latter is also described in HIV-2 infection. Conversely, positions 11 and 25 were not found associated with HIV-2 tropism prediction. However, the large genotypic differences between HIV-1 and HIV-2 V3 loops render it difﬁcult to make such comparisons.

Regarding viral tropism analysis of HIV-2 gp105 V3-loop se- quences with a known coreceptor usage available in the literature [16, 17, 30, 31], a sensitivity of  $65\%$  and a speciﬁcity of  $100\%$  in detecting D/M-X4 viruses was shown using the major genotypic determinants of CXCR4 usage we identiﬁed in our study. In all HIV-2 clinical isolates from our series, a mutation at residue 18 is sufﬁcient to predict D/M-X4 tropism. However, among the 7 X4 isolates from the previous study data set, 6 displayed a mutant residue at residue 18; the remaining 1 exhibited only a global net charge of  $^{+7}$  [16]. This suggests that residue 18 alone is probably not sufﬁcient to predict CXCR4 coreceptor usage. In this rean- alysis, the lower concordance was observed in samples exhibiting a D/M-tropic mixture, because 6 of the 10 reported D/M- tropic viruses were classiﬁed as R5 by analysis with the 4 major genotypic determinants. This observation may be due to the various phe- notypic tropism assays used in these studies and to the possible differences in double chromatogram peaks interpretation. Similar rates of sensitivity and speciﬁcity were obtained for tropism ge- notypic prediction in HIV-1 infection. Thus, in the ANRS Geno- TropismStudy,which evaluatedthe genotypicpredictionof HIV-1 coreceptor use versus the phenotypic Troﬁle assay, the highest sensitivity of genotypic-tropism prediction was  ${\sim}60\%$  according to different genotypic algorithms [38]. For all algorithms tested, the speciﬁcity to detect X4 compared with the Troﬁle test ranged from  $79.3\%$  to  $98.5\%$  [38]. Similar results of genotypic prediction compared to the standard Troﬁle assay have been reported after analysis of the MOTIVATE-1 study database [39].



In conclusion, in our study we established, for the ﬁrst time, a strong association between HIV-2 phenotypic tropism and V3-loop genotypic determinants based on a large series of pa- tients offering the opportunity to identify R5- and/or X4-tropic viruses. This genotypic tropism test could help improve the knowledge on HIV-2 pathogenesis by describing the evolution of tropism during the natural history of infection and also by describing the tropism in different compartments.

### Notes

Acknowledgments.C. C., F. D., S. M., F. B. V., and D. D. contributed to the study concept. B. V., A. S., and G. C. performed genotypic tests. B. V., M. H. N., A. S., and G. C. performed phenotypic tests. C. C., L. L., F. D., S. M., and D. D. contributed to the analysis and interpretation of the data. C. C., B. V., F. D., F. B. V., and D. D. contributed to the writing of the manuscript. All authors contributed to the critical reviewing of the manuscript.

Financial support.This work was supported by the French National Agency for Research on AIDS and Viral Hepatitis (ANRS) and by the European Community’s Seventh Framework Programme (grant FP7/ 2007–2013) under the project Collaborative HIV and Anti-HIV Drug Resistance Network (CHAIN) (grant 223131).

Potential conﬂicts of interest.All authors: No reported conﬂicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conﬂicts of Interest. Conﬂicts that the editors consider relevant to the content of the manuscript have been disclosed.

### References

1. Valadas E, Francxa L, Sousa S, Antunes F. 20 years of HIV-2 infection in Portugal: trends and changes in epidemiology. Clin Infect Dis 2009; 48:1166–7.

2. Brunet S, Thierry D, Barin F, et al. Surveillance de l’infection a\` VIH-sida en France, 2007. BEH 2008; 45:434–42.

3. Roquebert B, Damond F, Collin G, et al. HIV-2 integrase gene poly- morphism and phenotypic susceptibility of HIV-2 clinical isolates to the integrase inhibitors raltegravir and elvitegravir in vitro. J Antimicrob Chemother 2008; 62:914–20.

4. Ntemgwa ML, d’Aquin Toni T, Brenner BG, Camacho RJ, Wainberg MA. Antiretroviral drug resistance in human immunodeﬁciency virus type 2. Antimicrob Agents Chemother 2009; 53:3611–9.

5. Ren J, Bird LE, Chamberlain PP, Stewart-Jones GB, Stuart DI, Stammers DK. Structure of HIV-2 reverse transcriptase at 2.35-A resolution and the mechanism of resistance to non-nucleoside inhibitors. Proc Natl Acad Sci U.S.A 2002; 99:14410–5.

6. Witvrouw M, Pannecouque C, Switzer WM, Folks TM, De Clercq E, Heneine W. Susceptibility of HIV-2, SIV and SHIV to various anti-HIV-1 compounds: implications for treatment and postexposure prophylaxis. Antivir Ther 2004; 9:57–65.

7. Poveda E, Briz V, Soriano V. Enfuvirtide, the ﬁrst fusion inhibitor to treat HIV infection. AIDS Rev 2005; 7:139–47.

8. Desbois D, Roquebert B, Peytavin G, et al. In vitro phenotypic susceptibility of human immunodeﬁciency virus type 2 clinical isolates to protease inhibitors. Antimicrob Agents Chemother 2008; 52:1545–8.

9. Willey S, Peters PJ, Sullivan WM, Dorr P, Perros M, Clapham PR. Inhibition of CCR5-mediated infection by diverse R5 and R5X4 HIV and SIV isolates using novel small molecule inhibitors of CCR5: effects of viral diversity, target cell and receptor density. Antiviral Res 2005; 68:96–108.

10. Armstrong-James D, Stebbing J, Scourﬁeld A, et al. Clinical outcome in resistant HIV-2 infection treated with raltegravir and maraviroc. Antiviral Res 2010; 86:224–6.

11. Stegmann S, Manea ME, Charpentier C, et al. Foscarnet as salvage therapy in HIV-2-infected patient with antiretroviral treatment failure. J Clin Virol 2010; 47:79–81.

12. McKnight A, Dittmar MT, Moniz-Periera J, et al. A broad range of chemokine receptors are used by primary isolates of human immu- nodeﬁciency virus type 2 as coreceptors with CD4. J Virol 1998; 72:4065–71.

13. Mo¨rner A, Bjo¨rndal A, Leandersson A, Albert J, Bjo¨rling E, Jansson M. CCR5 or CXCR4 is required for efﬁcient infection of peripheral blood mononuclear cells by promiscuous human immunodeﬁciency virus type 2 primary isolates. AIDS Res Hum Retroviruses 2002; 18:193–200. 14. Guillon C, van der Ende ME, Boers PH, Gruters RA, Schutten M, Osterhaus AD. Coreceptor usage of human immunodeﬁciency virus type 2 primary isolates and biological clones is broad and does not correlate with their syncytium-inducing capacities. J Virol 1998; 72:6260–3.

15. Mo¨rner A, Bjo¨rndal A, Albert J, et al. Primary human immunodeﬁ- ciency virus type 2 (HIV-2) isolates, like HIV-1 isolates, frequently use CCR5 but show promiscuity in coreceptor usage. J Virol 1999; 73:2343–9.

16. Owen SM, Ellenberger D, Rayﬁeld M, et al. Genetically divergent strains of human immunodeﬁciency virus type 2 use multiple coreceptors for viral entry. J Virol 1998; 72:5425–32.

17. Shi Y, Brandin E, Vincic E, et al. Evolution of human immunodeﬁciency virus type 2 coreceptor usage, autologous neutralization, envelope se- quence and glycosylation. J Gen Virol 2005; 86:3385–96.

18. Blaak H, Boers PH, Gruters RA, Schuitemaker H, van der Ende ME, Osterhaus AD. CCR5, GPR15, and CXCR6 are major coreceptors of human immunodeﬁciency virus type 2 variants isolated from individuals with and without plasma viremia. J Virol 2005; 79:1686–700.

19. Zhang Y, Lou B, Lal RB, Gettie A, Marx PA, Moore JP. Use of inhibitors to evaluate coreceptor usage by simian and simian/human immunode- ﬁciency viruses and human immunodeﬁciency virus type 2 in primary cells. J Virol 2000; 74:6893–10.

20. Kuhmann SE, Hartley O. Targeting chemokine receptors in HIV: a status report. Annu Rev Pharmacol Toxicol 2008; 48:425–61.

21. Tsibris AM, Kuritzkes DR. Chemokine antagonists as therapeutics: focus on HIV-1. Annu Rev Med 2007; 58:445–59.

22. Whitcomb JM, Huang W, Fransen S, et al. Development and charac- terization of a novel single-cycle recombinant-virus assay to determine human immunodeﬁciency virus type 1 coreceptor tropism. Antimicrob Agents Chemother 2007; 51:566–75.

23. Jensen MA, van ’t Wout AB. Predicting HIV-1 coreceptor usage with sequence analysis. AIDS Rev 2003; 5:104–12.

24. Beerenwinkel N, Sing T, Lengauer T, et al. Computational methods for the design of effective therapies against drug resistant HIV strains. Bioinformatics 2005; 21:3943–50.

25. Web PSSM. http://indra.mullins.microbiol.washington.edu/webpssm/ index.html. Accessed 10 March 2011.

26. Geno2pheno coreceptor. http://coreceptor.bioinf.mpi-inf.mpg.de/ index.php. Accessed 10 March 2011.

27. Skrabal K, Low AJ, Dong W, et al. Determining human immunodeﬁ- ciency virus coreceptor use in a clinical setting: degree of correlation between two phenotypic assays and a bioinformatic model. J Clin Microbiol 2007; 45:279–84.

28. Raymond S, Delobel P, Mavigner M, et al. Correlation between geno- typic predictions based on V3 sequences and phenotypic determination of HIV-1 tropism. AIDS 2008; 22:F11–6.

29. Albert J, Sta˚lhandske P, Marquina S, et al. Biological phenotype of HIV type 2 isolates correlates with V3 genotype. AIDS Res Hum Retroviruses 1996; 12:821–8.

30. Isaka Y, Sato A, Miki S, et al. Small amino acid changes in the V3 loop of human immunodeﬁciency virus type 2 determines the coreceptor usage for CXCR4 and CCR5. Virology 1999; 264:237–43.

31. Kulkarni S, Tripathy S, Agnihotri K, et al. Indian primary HIV-2 isolates and relationship between V3 genotype, biological phenotype and cor- eceptor usage. Virology 2005; 337:68–75.

32. Davis HE, Rosinski M, Morgan JR, Yarmush ML. Charged polymers modulate retrovirus transduction via membrane charge neutralization and virus aggregation. Biophys J 2004; 86:1234–42.

33. Ho WZ, Cherukuri R, Ge SD, et al. Centrifugal enhancement of human immunodeﬁciency virus type 1 infection and human cytomegalovirus gene expression in human primary monocyte/macrophages in vitro. J Leukoc Biol 1993; 53:208–12.

34. Pietroboni GR, Harnett GB, Bucens MR. Centrifugal enhancement of human immunodeﬁciency virus (HIV) and human herpesvirus type 6 (HHV-6) infection in vitro. J Virol Methods 1989; 24:85–90.

35. Vo¨dro¨s D, Tscherning-Casper C, Navea L, Schols D, De Clercq E, Fenyo¨ EM. Quantitative evaluation of HIV-1 coreceptor use in the GHOST3 cell assay. Virology 2001; 291:1–11.

36. Cho MW, Lee MK, Carney MC, Berson JF, Doms RW, Martin MA. Identiﬁcation of determinants on a dualtropic human immunodeﬁ- ciency virus type 1 envelope glycoprotein that confer usage of CXCR4. J Virol 1998; 72:2509–15.

37. Schols D, Este´ JA, Cabrera C, De Clercq E. T-cell-line-tropic human immunodeﬁciency virus type 1 that is made resistant to stromal cell- derived factor 1alpha contains mutations in the envelope gp120 but does not show a switch in coreceptor use. J Virol 1998; 72:4032–7.

38. Recordon-Pinson P, Soulie´ C, Flandre P, et al. Evaluation of the genotypic prediction of HIV-1 coreceptor use versus a phenotypic assay and correlation with the virological response to maraviroc: the ANRS GenoTropism study. Antimicrob Agents Chemother 2010; 54:3335–40.

39. McGovern RA, Thielen A, Mo T, et al. Population-based V3 genotypic tropism assay: a retrospective analysis using screening samples from the A4001029 and MOTIVATE studies. AIDS 2010; 24:2517–25.
www.elsevier.com/locate/jhydrol

# The response of ﬂow duration curves to afforestation

Patrick N.J. Lane\*, Alice E. Best, Klaus Hickel, Lu Zhang

School of Forest and Ecosystem Studies, University of Melbourne, P.O. Box 137, Heidelberg, Victoria 3084, Australia CSIRO Division of Land and Water, Canberra, ACT, Australia Cooperative Research Centre for Catchment Hydrology, Canberra, ACT, Australia Department of Civil and Environmental Engineering, University of Melbourne, Victoria, Australia Received 1 October 2003; revised 22 December 2004; accepted 3 January 2005



### Abstract

The hydrologic effect of replacing pasture or other short crops with trees is reasonably well understood on a mean annual basis. The impact on ﬂow regime, as described by the annual ﬂow duration curve (FDC) is less certain. A method to assess the impact of plantation establishment on FDCs was developed. The starting point for the analyses was the assumption that rainfall and vegetation age are the principal drivers of evapotranspiration. A key objective was to remove the variability in the rainfall signal, leaving changes in streamﬂow solely attributable to the evapotranspiration of the plantation. A method was developed to (1) ﬁt a model to the observed annual time series of FDC percentiles; i.e. 10th percentile for each year of record with annual rainfall and plantation age as parameters, (2) replace the annual rainfall variation with the long term mean to obtain climate adjusted FDCs, and (3) quantify changes in FDC percentiles as plantations age. Data from 10 catchments from Australia, South Africa and New Zealand were used. The model was able to represent ﬂow variation for the majority of percentiles at eight of the 10 catchments, particularly for the 10–50th percentiles. The adjusted FDCs revealed variable patterns in ﬂow reductions with two types of responses (groups) being identiﬁed. Group 1 catchments show a substantial increase in the number of zero ﬂow days, with low ﬂows being more affected than high ﬂows. Group 2 catchments show a more uniform reduction in ﬂows across all percentiles. The differences may be partly explained by storage characteristics. The modelled ﬂow reductions were in accord with published results of paired catchment experiments. An additional analysis was performed to characterise the impact of afforestation on the number of zero ﬂow days  $(N_{\mathrm{zero}})$  for the catchments in group 1. This model performed particularly well, and when adjusted for climate, indicated a signiﬁcant increase in. The zero ﬂow day method could be used to determine change $N_{\mathrm{zero}}$  in the occurrence of any given ﬂow in response to afforestation. The methods used in this study proved satisfactory in removing the rainfall variability, and have added useful insight into the hydrologic impacts of plantation establishment. This approach provides a methodology for understanding catchment response to afforestation, where paired catchment data is not available.  2005 Elsevier B.V. All rights reserved. $@$  Keywords: Afforestation; Flow duration curves; Flow reduction; Paired catchments



## 1. Introduction

Widespread afforestation through plantation estab- lishment on non-forested land represents a potentially signiﬁcant alteration of catchment evapotranspiration (ET). Using data collated from multiple catchment studies, researchers have demonstrated a consistent difference in ET between forests and grass or short crops, and the relationship between ET and rainfall on a mean annual basis (Holmes and Sinclair, 1986; Vertessy and Bessard, 1999; Zhang et al., 1999, 2001). Once annual rainfall exceeds  $400–500\;\mathrm{mm}$ , there is an increasing divergence between forest and grassland ET (Zhang et al., 2001). Research from South Africa in particular has demonstrated ﬂow reduction following afforestation with both pine and eucalypt species (Bosch, 1979; Van Lill et al., 1980; Van Wyk, 1987; Bosch and Von Gadow, 1990; Scott and Smith, 1997; Scott et al., 2000). In regions, where water is an increasingly valuable resource, prediction of the long-term hydrologic impact of afforestation is a prerequisite for the optimal planning of catchment land use.

Zhang et al. (1999, 2001) developed simple and easily parameterised models to predict changes in mean annual ﬂows following afforestation. However, there is a need to consider the annual ﬂow regime as the relative changes in high and low ﬂows may have considerable site speciﬁc and downstream impacts.. Sikka et al. (2003) recently showed a change from grassland to Eucalyptus globulus plantations in India decreased a low ﬂow index by a factor of two during the ﬁrst rotation (9 years), and by 3.75 during the second rotation, with more subdued impact on peak ﬂows. The index was deﬁned as the 10 day average ﬂow exceeded  $95\%$  of the time, obtained from analysis of 10-day ﬂow duration curves. Scott and Smith (1997) reported proportionally greater reductions in low ﬂows (75–100th percentiles) than annual ﬂows from South African research catchments under conversions from grass to pine and eucalypt plantations, while Bosch (1979) found the greatest reduction in seasonal ﬂow from the summer wet season. Fahey and Jackson (1997) reported the reduction in peak ﬂows was twice that of total ﬂow and low ﬂows for pine afforestation in New Zealand. The generalisations that can be drawn from annual analyses, where processes and hydrologic responses are to a certain extent integrated may not apply on a seasonal or shorter scale. Further, the observed impacts of any land use change on ﬂows may be exaggerated or understated depending on the prevailing climate. Observations of ﬂow during extended wet or dry spells, or with high annual variability can obscure the real impacts. Fig. 1 plots annual FDCsover 12 years of plantation growth for one of the catchments used in this study, Pine Creek. The net change in ﬂow is obscured by rainfall variability; e.g. the greatest change in the FDC is in 1996, with the stream ﬂowing  $<20\%$  of the time. This may be compared with 2000, where there is substantially higher ﬂows.



This paper presents the results of a project aimed at quantifying changes in annual ﬂow regime of catchments following plantation establishment. The ﬂow regime is represented by the ﬂow duration curve (FDC). The key assumption was that rainfall and forest age are the principal drivers of evapotranspira- tion. For any generalisation of response of the FDC to vegetation change, the variation in the annual climate signal must be removed. The time-tested solution to this problem is the paired-catchment (control versus treatment) experiment. The beneﬁts in such studies are manifold: unambiguous measures of trends, insights into the processes driving those trends, excellent opportunities for model parameterisation and validation. However these data are not readily available for the range of treamtments and environ- ments required. Consequently, the aims of this project were to (1) ﬁt a model to the observed annual time series of FDC percentiles; i.e. 10th percentile for each year of record with annual rainfall and plantation age as parameters, (2) replace the annual rainfall variation with the long term mean to obtain climate adjusted FDCs, and (3) quantify changes in FDC percentiles as plantations age. If the climate signal, represented by rainfall, could be successfully removed, the resulting changes in the FDC would be solely attributable to the vegetation.



## 2. Methods

## 2.1. Characterisation of ﬂow regime

Flow duration curves display the relationship between streamﬂow and the percentage of time the streamﬂow is exceeded as a cumulative density function They can be constructed for any time period (daily, weekly, monthly, etc.) and provide a graphical and statistical view of historic streamﬂow variability in a single catchment or a comparison of inter- catchment ﬂow regimes. Vogel and Fennessey (1994) and Smakhtin (1999, 2001) demonstrate the utility (and caveats) of FDCs in characterising, comparing and predicting ﬂow regimes at varying temporal scales. Fig. 1 is an example of annual FDCs constructed from daily ﬂows. For the consideration of annual ﬂow regime, daily ﬂows are an appropriate time step for FDC construction.

FDCs were computed from the distribution of daily ﬂows for each year of record based on the appropriate water years (May–April or November–October) for 10 Southern Hemisphere catchments. Each 10th percentile (decile) was extracted from the annual FDCs of each catchment to form the data sets for analysis. For the purpose of characterising changes in each of the deciles, it is assumed that the time series is principally a function of climate and vegetation characteristics. Given rainfall is generally the most important factor affecting streamﬂow and the most easily accessed data, it is chosen to represent the climate. Catchment physical properties such as soil properties and topography are assumed to be time invariant and therefore their impact on runoff is considered constant throughout the analysis. As trees intercept and transpire at increasing rates until canopy closure, a time term is required to represent plantation growth. A simple model relating the time series of each decile with rainfall and vegetation characteristics can be expressed as:



## 
$$
Q_{\%}=f(P)+g(T)
$$


(1) where  $Q_{\mathscr{V}}$  is the percentile ﬂow, is a function of $f(P)$  rainfall and  $g(T)$  is a function of the age of the plantation. Annual rainfall was chosen as the rainfall statistic as it proved to be the most robust predictor of ﬂow over the whole range of ﬂow percentiles, as compared with rainfall percentiles; e.g. median rain- fall versus 10th ﬂow percentile. The use of annual rainfall also minimises parameter complexity. The choice of model form is dependent on selecting a function that describes the relationship between forest age and ET. Scott and Smith (1997) demonstrated cumulative reductions in annual and low ﬂows resulting from afforestation ﬁtted a sigmoidal function, similar to forest growth functions. Conse- quently, we used a sigmoidal function to characterise the impact of plantation growth on each ﬂow decile. Fig. 2a is a schematic of the change in the FDC over time. The model took the form:



## 
$$
Q_{\%}=a+b(\Delta P)+\frac{Y}{1+\exp\!\left(\frac{T-T_{\mathrm{half}}}{S}\right)}
$$


(2) where  $Q_{\mathscr{V}}$  is the percentile ﬂow (i.e.  $Q_{50}$  is the 50th percentile ﬂow),  $Y$  and  $S$  are coefﬁcients of the sigmoidal term,  $\Delta P$  is the deviation of annual rainfall from the period of record average, and  $T_{\mathrm{half}}$  is the time in years at which half of the reduction in  $Q_{\mathbb{Z}}$  due to afforestation has taken place. For the average climate condition  $\Delta P\!=\!0$ ,  $a$  becomes the value of  $Q_{\%}$  when the new equilibrium plantation water use under afforestation is reached. Y then gives the magnitude of change due to afforestation, and  $S$  describes the shape of the response as shown in Fig. 2b. For the average pre-treatment condition  $\Delta P\!=\!0$  at  $T{=}0$ ,  approximately equals $Q_{\%}$   $a+Y.$ . Estimation of a pre- afforestation condition would not require the time term. Details of the optimisation scheme and sensitivity tests on initial parameter values are given in Lane et al. (2003).



## 2.2. Zero ﬂow day analysis

A notable feature of Fig. 1 is the increase in the number of zero ﬂow days. A similar approach to Eq. (2), using an inverse sigmoidal function was employed to assess the impact of afforestation on the number of zero ﬂow days per year  $(N_{\mathrm{zero}})$ . In this case, the left hand side of Eq. (2) is replaced by  $N_{\mathrm{zero}}$ , and  $b$  and  $S$  are constrained to negative as decreases as $N_{\mathrm{zero}}$  rainfall increases, and increases with plantation growth:

## 
$$
N_{\mathrm{zero}}=a+b(\Delta P)+\frac{Y}{1+\exp\!\left(\frac{T-T_{\mathrm{half}}}{S}\right)}
$$


For the average pre-treatment condition  $\Delta P\!=\!0$  and  $T\!\!=\!0,\ N_{\mathrm{zero}}$  approximately equals  $a$ .  $Y$  gives the magnitude of change in zero ﬂow days due to afforestation, and  $S$  describes the shape of the response. For the average climate condition  $\Delta P\!=\!0$ ,  becomes the number of zero ﬂow days when the $a+Y$  new equilibrium condition under afforestation is reached.



## 2.3. Statistical analyses

The coefﬁcient of efﬁciency  $(E)$  (Nash and Sutcliffe, 1970; Chiew and McMahon, 1993; Legates and McCabe, 1999) was used as the ‘goodness of ﬁt’ measure to evaluate the ﬁt between observed and predicted ﬂow deciles (2) and zero ﬂow days (3). is $E$  given by:

## 
$$
E=1.0-\frac{\sum_{i=1}^{N}(O_{i}-P_{i})^{2}}{\sum_{i=1}^{N}(O_{i}-\bar{O})^{2}}
$$


(4) where  $o$  are observed data,  $P$  are predicted values, and  $\bar{O}$  is the mean for the entire period.  $E$  is unity minus the ratio of the mean square error to the variance in the observed data, and ranges from  $-\infty$  to 1.0. Higher values indicate greater agreement between observed and predicted data as per the coefﬁcient of determination  $(r^{2})$ .  $E$  is used in preference to  $r^{2}$  in evaluating hydrologic modelling because it is a measure of the deviation from the 1:1 line. As  $E$  is always  $<r^{2}$ we have arbitrarily considered to $E\!>\!0.7$  indicate adequate model ﬁts.



It is important to assess the signiﬁcance of the model parameters to check the model assumptions that rainfall and forest age are driving changes in the FDC. The model (2) was split into simpliﬁed forms, where only the rainfall or time terms were included by setting, as shown in Eq. (5), or $b\!=\!0$   $Y{=}0$  as shown in Eq. (6). The component models (5) and (6) were then tested against the complete model, (2).

## 
$$
Q_{\%}=a+{\frac{Y}{1+\exp\!\left({\frac{T-T_{\mathrm{half}}}{S}}\right)}}
$$


(5) and



## 
$$
Q_{\%}=a+b\Delta P
$$


(6)

For both the ﬂow duration curve analysis and zero ﬂow days analysis, a  $t$ -test was then performed to test whether (5) and (6) were signiﬁcantly different to (2). A critical value of  $t$  exceeding the calculated  $t\cdot$ -value when comparing (5) and (2) would indicate the time term in (6) was required to improve the complete model and is therefore signiﬁcant, and vice versa.



Due to the constraint that the rainfall and time term must be positive, a one tailed  $t\cdot$ -test was applied. The -value was calculated as $t\mathrm{.}$   $F^{0.5}$ , and compared with the critical value for signiﬁcance at the 0.05 level. The  $F.$ -statistic was calculated as:

## 
$$
F=\mathrm{\frac{[(SSE_{s}-S S E_{c})/(d f_{c}-d f_{s})]}{S S E_{c}/d f_{c}}}
$$


(7) where SSE is the residual sum of the squared errors, df is degrees of freedom, and the subscripts s and c refer to the simpliﬁed model and complete models, respectively.



## 3. Data sets

Daily streamﬂow data were obtained from 10 catchment studies from southeastern Australia, New Zealand and South Africa. The initial criteria for selection of these catchments were a known veg- etation history and streamﬂow records of good quality. The ideal data sets were those with a lengthy pre- and post-treatment (plantation establishment) ﬂow record with approximately  $100\%$  of the catch- ment converted from grassland or a crop equivalent to plantation. In reality, all these criteria were not easy to satisfy. For example in Victoria, Australia, the best data is from Stewarts Creek, a set of decommissioned research catchments with 9 years of pre-treatment data and 25 years of post-treatment. Here, though, the treatment was a conversion from native eucalypt forest to pine. The assumption made for this data set is that the immediate post-treatment period may be viewed as a non-forested condition. This condition is likely to approximate the ET conditions of pasture or short crops for up to 3 years. Catchment details and treatments are given in Table 1.

All catchments, with the exception of Traralgon Creek, were afforested with pine species, predomi- nantly Pinus radiata, with $P.$ . patula planted at the two Cathedral Peak catchments. Traralgon Creek has only  $6\%$  pine, with the remainder eucalypts species, most of which is Eucalyptus regnans.

 indicates that the rainfall term was signiﬁcant at the  $P$  $5\%$  level,  $T$  indicates that the time term was signiﬁcant at the  $5\%$  level, \* represents signiﬁcance at the  $10\%$  level, and na denotes too few data points for meaningful analysis.

Data on soil characteristics have been obtained from published reports and personal communication with researchers, but is far from uniform, particularly regarding porosity. Consequently only an indication of mean depth is reported here. However, this does give some indication of the likely relative storage capacities of the catchments. To obtain insights into the pre-afforestation hydrologic characteristics a baseﬂow separation was performed on the daily ﬂows for the ﬁrst 3 years following disturbance, using the digital ﬁltering method of Lyne and Hollick (1979) with a ﬁlter coefﬁcient of 0.925 and three passes. The resultant average baseﬂow index (BFI), the ratio of baseﬂow to total ﬂow, is given in Table 1. The Australian catchments displaya notably lower BFI than the South African and New Zealand catchments.ForStewartsCreek,PineCreek and Redhill the lower BFI is matched by the shallow soils. Pre-treatment data is not available for all catchment in the data set, so it was decided for the sake of consistency in the analysis to start each of the data sets in the year of treatment. The FDCs were constructed for water years of May–April for eight catchments. The 2 Cathedral Peak catchments were analysed for November–October because of the summer rainfall maxima (Table 2).



## 4. Results

## 4.1. Model evaluation

The ﬁt of the complete model, Eq. (2), to the observed data was generally good. Table 2 gives

ns Indicates that no solution was found, and na denotes deciles with too few data points for analysis.

the coefﬁcient of efﬁciency  $(E)$  for each ﬂow percentile at all the catchments. The majority of ﬁts  $(77\%)$  returned  $E\!>\!0.7$ , with  $60\%~0.8$  or better. The signiﬁcance of the rainfall and time terms is given in Table 3 for all deciles, where solutions were found. There were not enough data to ﬁt the model in ﬁve instances because of extended periods of zero ﬂows. This problem is addressed to some extent in the zero ﬂow analysis. If the rainfall signal is to be separated from the vegetation signal the rainfall terms must be signiﬁcant. This term,  $b$ , was signiﬁcant for  $75\%$  of the deciles at the 0.05 level, and a further  $9\%$  at the 0.10 level. The incidence of signiﬁcance was greatest for the 10–50th percentiles at 45 of the 50 data sets at the 0.05 level. The time term,  $Y$  returned similar results, with  $80\%$  of the deciles signiﬁcant at 0.05 level. There were an additional  $9\%$  of deciles signiﬁcant at the 0.10 level.

The poorest  $E$  values were those from Lambrechts- bos A and B. The high  $E$  for 50–100th deciles at Biesievlei, where  $b$  was not signiﬁcant are notable. In general the model ﬁts the higher ﬂows (lower deciles) better, most of the poorer ﬁts are in the 80–100 percentile range. This can be expected given the results of the signiﬁcance tests for  $b$ . The results of the sensitivity analysis suggested that the  $E$  values for Glendhu 2 and for 10th and 20th percentiles from Cathedral Peak 3 may exaggerate the goodness of ﬁt to the exact form of the model (Lane et al., 2003).

## 4.2. Adjusted FDCs—magnitude of ﬂow reductions

Following the successful ﬁtting of (2) to the observed percentiles, the FDCs were adjusted for climate by setting  $\Delta P$  to zero, representing long term average annual rainfall. The climate adjusted FDCs produce an estimation of the change in ﬂow percentiles over time for each catchment due to afforestation that may be viewed in two forms: new FDCs, adjusted for climate, as exempliﬁed in Fig. 3 for Stewarts Creek 5, and a comparison between all catchments of the maximum change in yield (given by  for each ﬂow percentile from baseline ﬂows (given $Y)$  by  $a+Y)$  as shown in Fig. 4. Where the new equilibrium of maximum water use is reached, the adjusted FDCs for individual years should be identical if rainfall variability has been accounted for. The new equilibrium is approximately reached for  $T{=}2T_{\mathrm{half}}$ .

 values are given in $T_{\mathrm{half}}$  Table 4. Fig. 3 shows that for most deciles the adjusted FDCs are identical for 12 and 20 years after treatment. This ﬁgure clearly demonstrates the necessity for FDC adjustment, particularly for the 20 years FDC.

The relative net ﬂow change due to afforestation is given by  $Y/(Y+a)$ , which represents the change from the old equilibrium water use condition of pre-treatment vegetation to the new equilibrium condition at forest canopyclosure.Thisquantityisplottedforallcatchments in Fig. 4. Some deciles have been removed from the data set, the 10th and 50th percentile for Glendhu 2 and the 10th and 20th percentiles from Cathedral Peak 3. The optimised value ofwaszero or near zerofor these cases, $a$  which is not consistent with the conceptual model. The changes shown in Fig. 4 are variable. However, there are some commonalities between catchment responses. Two types of responses (groups) were identiﬁed. Group 1 catchments show a substantial increase in the number of zero ﬂow days, with a greater proportional reduction in low ﬂows than high ﬂows. Group 2 catchments show a more uniform proportional reduction in ﬂows across all percentiles, albeit with some variability. The catchments in each group are: Group 1: Stewarts Creek, Pine Creek, and Redhill Group 2: Cathedral Peak 2 and 3, Lambrechtsbos A, Lambrechtsbos B, Glendhu 2, Biesievlei and Traralgon Creek Group 1 exhibit both the highest reduction of ﬂows overall, and show the largest proportional reduction at lower ﬂows, leading to a complete cessation of ﬂow. Comparison of ﬂow reductions is hindered slightly by the range of afforestation at the catchments (Table 1). These results could be scaled up to  $100\%$  afforested if it is assumed there is a linear relationship between the area planted and ﬂow reductions. As there is no evidence that this is the case we have not presented scaled reductions here. Linear scaling would shift the reduction curves upward for those catchments that are less than  $100\%$  afforested, but would not change the shape of the curves or our groupings.

 



## 4.3. Timing of ﬂow reductions

The speed of ﬂow responses to afforestation can be evaluatedbyexaminingthevalueof( $T_{\mathrm{half}}$ Table4).There is substantial variation in response times both over the percentile spread in some individual catchments, and between the catchments. The majority of responses have a  $T_{\mathrm{half}}$  value between 5 and 10 years. Pine Creek and StewartsCreek,RedhillandLambrechtsbosAexhibitthe fastest responses, with Biesievlei showing the most uniformly slow response.  $T_{\mathrm{half}}$  for the South African catchments display a good correspondence to published annual changes (Scott et al., 2000; Van Wyk, 1987), excepting the 10–20th deciles for both Cathedral Peak

Note that no solution could be found for the 50 percentile for Glendhu indicted by the ns.

a Rainfall refers to the rainfall in the year used for comparison of results. The value in brackets refers to the deviation from the mean annual rainfall for the period of record. b Total ﬂow reduction calculated by $\textstyle\sum Y/\sum(a+Y)$  $\textstyle\sum Y/\sum(a+Y)$  for all deciles. c Low ﬂow reduction calc $a$ ated  $Y$ y P Y= PðaCYÞ d For Cathedral Peak 3 the a and Y values for the 10 and 20th percentiles were excluded as the values of  $a$  were lower then the values of the 30–100th percentiles.

catchments and the lower deciles at Lambrechtsbos B. The  $T_{\mathrm{half}}$  from Glendhu 2 appears to be substantially lower than other published data (Fahey and Jackson, 1997).

## 4.4. Comparison with paired catchment studies

A further check on the overall model performance is a comparison with published results of paired catchment studies. The data that can be compared with our results are presented in Table 5 and can be broadly compared with Fig. 4. These data are reductions in years with near average annual rainfall, and at a time after treatment when maximum changes in streamﬂow have occurred. Table 5 also includes estimates on the total and low ﬂow reductions calculated from this study. Results from Pine Creek and Traralgon Creek are not included in Table 5 as these catchments are not paired. Exact comparisons are impossible because of the rainfall variability, and lack of calibration period for Redhill. Despite this, Table 5 shows that total and low ﬂow reductions estimated from our study are comparable to the results from paired catchment studies, indicating that our simple model has successfully removed the rainfall signal.

## 4.5. Zero ﬂow days

As this analysis could only be applied, where there was consistent drying up of streams, it was conﬁned to Stewarts Creek, Pine Creek and Redhill catchments. The model returned values of  $E$  of 0.95, 0.99 and 0.99, respectively. The  $t\cdot$ -tests on  $b$  and  $\Delta N_{\mathrm{zero}}$  returned signiﬁcant results at the 0.05 level for both parameters at all three catchments. The climate adjusted zero ﬂow days are shown in Fig. 5. The increases in zero ﬂow days are substantial with ﬂows conﬁned to less than  $50\%$  of the time by year 8 at Stewarts $\mathrm{Ck}$  and Pine Ck and year 11 at Redhill. The latter has changed from an almost permanent to a highly intermittent stream. The curves are also in sensible agreement with the ﬂow reductions in Fig. 4.

## 5. Discussion

The aims of the project have largely been met. The general characterisation of FDCs and adjustment for climate has been very encouraging given the task of ﬁtting our model to 10 ﬂow percentiles, for 10 different catchments (resulting in 100 model ﬁts) with substantially varying spatial scales, soils and geology, species planted and climatic environments. Although there were poor results for individual deciles, the FDCs at eight of the 10 catchments were adequately described by Eq. (2). The results of the statistical tests in which the rainfall term was signiﬁcant for most deciles demon- strated the model structure was appropriate for adjusting the FDCs for climatic (rainfall) variability. The comparisons of our results with published paired catchment analyses are satisfactory, although the different methodologies make direct comparisons of deciles with total ﬂow uncertain. Low ﬂows at Lambrechtsbos  $\mathbf{B}$  appear to be over-estimated by our model, which is unsurprising as the model ﬁt was poor. The remaining four South African catchments, and also Redhill and Stewarts Creek are in good agreement with the published values, particularly when the deviation of average rainfall is considered. Glendhu 2 reductions are close to the reported  $27\%$ , but our model produces a heavier impact on the lower ﬂows. Overall, it appears there are no signiﬁcant discrepancies with the published paired catchment analyses. We suggest our technique represents an alternative to the paired-catchment method for assessing hydrologic response to vegetation treat- ment, where paired data are unavailable. The method has not yet resulted in a predictive model, but has increased our knowledge of afforestation impacts. This is a valuable outcome given the contentious issue of afforestation in Australia and other countries, and a current paucity of data on inter-annual ﬂows. It should be noted that nine of the 10 catchment were pine species. More data is required to compare the impact of hardwood species, particularly eucalypts, on the FDC. Unfortunately these data are currently scarce. There are substantial data on the physiological controls of eucalypt water use (see Whitehead and Beadle, 2004), but not at the catchment scale.

 



The model ﬁts show we have quantiﬁed the net impact of afforestation for the majority of the ﬂow percentiles in the 10 catchments. Results for the 10–50th percentiles were particularly encouraging. It is not surprising that the relationship between rainfall and ﬂow diminishes at lower ﬂows (60–100th percentile), where seasonal storage effects and rainfall distribution become more important drivers for runoff generation. The poorest model ﬁts were gained for Lambrechtsbos A and B. The likely reason at Lambrechtsbos A is an observed annual decrease in stand water use after 12 years (Scott et al., 2000) which does not conform to the sigmoidal form of our model over the full 19 years of record. The failure of the model to ﬁt the lower ﬂows at Lambrechtsbos  $\mathrm{B}$  is not as explicable. A decrease in stand water use in this catchment is observed as the plantation ages, but does not occur during the ﬁrst 20 years after treatment (Scott et al., 2000). Other data from South Africa (Scott et al., 2000) indicate there are diminished ﬂow reductions as plantations age, but again, generally after 20 years. Our use of an asymptotic curve assumes a new equilibrium of stand water use is reached. The results of the model ﬁtting generally justify this assumption for the length of commercial plantation growth (up to 20 years) considered here. The physio- logical relationship between stand age and water use for plantation species other than  $E.$ . regnans have not been thoroughly investigated, although Cornish and Vertessy (2001) and Roberts et al. (2001) have shown young mixed species eucalypt forests may use more water than mature stands, and Putahena and Cordery (2000) suggest maximum Pinus radiata water use may have been reached after 12 years, with a subsequent decline.



The small Australian catchments converted to pine in response group 1 (Stewarts Creek 5, Pine Creek and Redhill) have similar shallow soils, potential evapo- transpiration and rainfall distribution (relatively uni- form) although Stewarts Creek is signiﬁcantly wetter. The combination of small catchment area and the increased transpirative demand that exceeds summer and autumn rainfall and stored water results in the large impact on lower ﬂows, compared to high ﬂows.

The magnitude of the response within Group 2 varies considerably, with greater reduction in ﬂows in the two Cathedral Peak catchments, and Lambrechtsbos B. Potential evaporation is in phase with rainfall at the Cathedral Peak sites as they receive  $85\%$  ( $1260\,\mathrm{mm}$  on average) of their rainfall in summer. The conjunction of peak demand and plant water availability may explain the high reductions relative to the remaining catchments in Group 2. In addition, the stocking density was described as ‘abnormally dense’ by Scott et al. (2000). Growth at Glendhu 2 was notably slow (Fahey and Jackson, 1997) and Lambrechtsbos A and Biesievlei are described as being within sub optimal growth zones (Scott and Smith, 1997) characterised by these authors as having relatively slow response times and lesser reductions that those at more optimal sites.

The response groups may be in part explained by the storage characteristics of the catchments. Accurate measures of storage are not available from the literature, but the soil depths and the baseﬂow index (Table 1) both show the three south eastern Australian catchments with the greatest reduction are likely to have the lowest storage capacity. The greater ﬂow reductions, particu- larly for low ﬂows, could be expected under these conditions. Inclusion of a storage term in the model is an obvious option for improving the analysis. However the addition of extra parameters would be at the cost of maintaining model simplicity, particularly as character- ising a transient storage is not trivial.

Traralgon Creek would be expected to have both the most subdued ﬂow reductions and longer response time because of the large area of  $E.$ . regnans forest, and uncertain vegetation record. Peak stand water use of a natural stand of this species is around 30 years. Additionally in this large, ‘real world’ catchment, there is a continuous cycle of forest management which includes harvesting. A mixture of pasture and ‘scrub’, which could represent signiﬁcant understorey stands, were replaced by plantation species. Conse- quently the difference between pre and post treatment ET may be less than at other catchments. Reductions of this magnitude could be more readily expected in larger, multi land use catchments than the very high impacts estimated at the smaller Australian catchments.

The analysis of zero ﬂow days was successful, demonstrating that the impact on ﬂow intermittence can be evaluated without of the entire FDC. This was helpful as the change in the higher percentiles (low ﬂows) could not always be modelled. The results for the three catchments analysed are a rather stark indication of the potential for highly increased zero ﬂow periods in small catchments, at least in south-eastern Australia. However, it should be noted these curves probably represent a maximum response as they are all derived from small catchments with small storage capacities and large percentages of afforestation. This method could be used to determine change in the occurrence of any given ﬂow in response to afforestation; e.g. to determine the likelihood of maintaining a reservoir storage or an environmental ﬂow that requires an average critical ﬂow.

## 6. Summary and conclusions

Thisprojectsoughtto(i)developamethodtoremove the climate signal from streamﬂow records to identify the impact of vegetation on ﬂow from afforested catchments, and (ii) quantify this impact on the ﬂow duration curve. A simple model was proposed that considered the age of plantation and the annual rainfall to be the principal drivers for evapotranspiration. This model was ﬁtted to the observed deciles of the FDC, and the climate signal was then removed from the stream- ﬂow records by adjusting the FDC for average rainfall over the period of record. The model was tested and applied to 10 afforested catchments. We successfully ﬁtted our model to catchments with varying spatial scales, species and environments, and haveshown thatit provides a means of separating the inﬂuence of climate and vegetation on the FDCs. The modelled results showed the greatest proportional impacts were for median and lower ﬂows. The ﬂow reductions from the three small catchments SE Australian were the highest and may reﬂect lower storages. The characterisation of the number of zero ﬂow days was also successful for these catchments in indicating a signiﬁcant increase in zero ﬂows. The ﬂow reductions identiﬁed here probably represent a maximum effect given the size of the catchments, level of afforestation and the shallow soils. These results have yielded useful new insights on the contentious issue of the hydrological impact of afforestation. This research has led to the development of a method to assess the net impact of afforestation on the ﬂow duration curve which does not require paired- catchments to remove climatic variability.



## Acknowledgements

The authors would like to thank Rory Nathan, Narendra Tuteja, Tom McMahon, Geoff Podger, Rob Vertessy, Glen Walker and Peter Hairsine for particu- larly helpful discussions on methodologies and reviews, Richard Morton for valuable statistical advice, Dave Scott for supplying the South African data, Barry Fahey for the New Zealand data, and Hancocks Victorian Plantations for vegetation data. The study was funded by the Victorian Department of Natural Resources and Environment Private Forestry Unit, the CRC for Catchment Hydrology, and the MDBC funded project ‘Integrated assessment of the effects of land use changes on water yield and salt loads’ (D2013).

## References

Bosch, J.M., 1979. Treatment effects on annual and dry period streamﬂow at Cathedral Peak. South African Forestry Journal 108, 29–37.

Bosch, J.M., Von Gadow, K., 1990. Regulating afforestation for water conservation in South Africa. Suid-Afrikaanse Bosboutydskrif 153, 41–54.

Chiew, F.H.S., McMahon, T.A., 1993. Assessing the adequacy of catchment streamﬂow yield estimates. Australian Journal of Soil Research 31, 665–680.

Cornish, P.M., Vertessy, R.A., 2001. Forest age-induced changes in evapotranspiration and water yield in a eucalypt forest. Journal of Hydrology 242, 43–63.

Fahey, B., Jackson, R., 1997. Hydrological impacts of converting nativeforestsandgrasslandstopineplantations,South Island, New Zealand. Agricultural and Forest Meteorology 84, 69–82.

Hickel, K., 2001. The effect of pine afforestation on ﬂow regime in small upland catchments. Masters Thesis, University of Stuttgart, p. 134.

Holmes, J.W., Sinclair, J.A., 1986. Water yield from some afforested catchments in Victoria. In Hydrology and Water Resources Symposium, Grifﬁth University, Brisbane 25–27 November 1986, pp. 214–218.

Lane, P.N.J., Best, A.E., Hickel, K., Zhang, L., 2003. The effect of afforestation on ﬂow duration curves. Cooperative Research Centre for Catchment Hydrology Technical Report 03/13, p. 25.

Legates, D.R., McCabe, G.J., 1999. Evaluating the use of ‘goodness- of-ﬁt’ measures in hydrologic and hydroclimatic model validation. Water Resources Research 35, 233–241.

Lyne, V.D., Hollick, M., 1979. Stochastic time-varying rainfall–runoff modelling. Hydrology and Water Resources Symposium, Perth. Institution of Engineers, Australia, pp. 89–92.

Nandakumar, N., Mein, R.G., 1993. Analysis of paired catchment data to determine the hydrologic effects of changes in vegetative cover on yield. Technical Report for Project UM010, Monash University Dept. of Civil Engineering, 219 p.

Nash, J.E., Sutcliffe, J.V., 1970. River ﬂow forecasting through conceptual models, I, A discussion of principals. Journal of Hydrology 10, 282–290.

Putahena, W.M., Cordery, I., 2000. Some hydrological effects of changing forest cover from eucalyptus to Pinus radiata. Agricul- tural and Forest Meteorology 100, 59–72.

Roberts, S., Vertessy, R.A., Grayson, R.G., 2001. Transpiration from Eucalyptus sieberi (L. Johnson) forests of different age. Forest Ecology and Management 143, 153–161.

Scott, D.F., Smith, R.E., 1997. Preliminary empirical models to predict reductions in total and low ﬂows resulting from afforestation. Water S.A. 23, 135–140.

Scott,D.F.,Prinsloo,F.W.,Moses,G.,Mehlomakulu,M., Simmers, A.D.A., 2000. Area-analysis of the South African catchmentafforestationexperimentaldata.WRCReport No. 810/1/00.

Sikka, A.K., Samra, J.S., Sharda, V.N., Samraj, P., Lakshmanan, V., 2003. Low ﬂow and high responses to converting natural grassland into bluegum (Eucalyptus globulus) in Ningiris watersheds of South India. Journal of Hydrology 270, 12–26.

Smakhtin, V.U., 1999. A concept of pragmatic hydrological time series modelling and its application in South African context. In Ninth South African National Hydrology Symposium, 29–30 November 1999, pp. 1–11.

Smakhtin, V.U., 2001. Low ﬂow hydrology: a review. Journal of Hydrology 240, 147–186.

Van Lill, W.S., Kruger, F.J., Van Wyk, D.B., 1980. The effect of afforestation with Eucalyptus grandis Hill ex Maiden and Pinus patula Schlect. et Cham. on streamﬂow from experimental catchments at Mokubulaan, Transvaal. Journal of Hydrology 48, 107–118.



Van Wyk, D.B., 1987. Some effects of afforestation on streamﬂow in the Western Cape Province, South Africa. Water S.A. 13, 31–36.

Vertessy, R.A., Bessard, Y., 1999. Anticipating the negative hydrologic effects of plantation expansion: results from a GIS-based analysis on the Murrumbidgee Basin, in: Croke, J., Lane, P.N.J. (Eds.), Forest Management for Water Quality and Quantity: Proceedings of the 2nd Erosion in Forests Meeting Cooperative Research Centre for Catchment Hydrology, Report 99/6, pp. 69–73.

Vogel, R.M., Fennessey, N.M., 1994. Flow duration curves. 1. New interpretation and conﬁdence intervals. Journal of Water Planning and Management 120 (4), 485–504.

Whitehead, D., Beadle C.L., 2004. Physiological regulation of productivity and water use in Eucalyptus: a review. Forest Ecology and Management, 193, 113–140.

Zhang, L., Dawes, W.R., Walker, G.R., 1999. Predicting the effect of vegetation changes on catchment average water balance. Coop- erative Research Centre for Catchment Hydrology Technical Report 99/12, p. 35.

Zhang, L., Dawes, W.R., Walker, G.R., 2001. Response of mean annual evapotranspiration to vegetation changes at catchment scale. Water Resources Research 37, 701–708.
www.elsevier.com/locate/jhydrol

# The response of ﬂow duration curves to afforestation

Patrick N.J. Lane\*, Alice E. Best, Klaus Hickel, Lu Zhang

School of Forest and Ecosystem Studies, University of Melbourne, P.O. Box 137, Heidelberg, Victoria 3084, Australia CSIRO Division of Land and Water, Canberra, ACT, Australia Cooperative Research Centre for Catchment Hydrology, Canberra, ACT, Australia Department of Civil and Environmental Engineering, University of Melbourne, Victoria, Australia Received 1 October 2003; revised 22 December 2004; accepted 3 January 2005



### Abstract

The hydrologic effect of replacing pasture or other short crops with trees is reasonably well understood on a mean annual basis. The impact on ﬂow regime, as described by the annual ﬂow duration curve (FDC) is less certain. A method to assess the impact of plantation establishment on FDCs was developed. The starting point for the analyses was the assumption that rainfall and vegetation age are the principal drivers of evapotranspiration. A key objective was to remove the variability in the rainfall signal, leaving changes in streamﬂow solely attributable to the evapotranspiration of the plantation. A method was developed to (1) ﬁt a model to the observed annual time series of FDC percentiles; i.e. 10th percentile for each year of record with annual rainfall and plantation age as parameters, (2) replace the annual rainfall variation with the long term mean to obtain climate adjusted FDCs, and (3) quantify changes in FDC percentiles as plantations age. Data from 10 catchments from Australia, South Africa and New Zealand were used. The model was able to represent ﬂow variation for the majority of percentiles at eight of the 10 catchments, particularly for the 10–50th percentiles. The adjusted FDCs revealed variable patterns in ﬂow reductions with two types of responses (groups) being identiﬁed. Group 1 catchments show a substantial increase in the number of zero ﬂow days, with low ﬂows being more affected than high ﬂows. Group 2 catchments show a more uniform reduction in ﬂows across all percentiles. The differences may be partly explained by storage characteristics. The modelled ﬂow reductions were in accord with published results of paired catchment experiments. An additional analysis was performed to characterise the impact of afforestation on the number of zero ﬂow days  $(N_{\mathrm{zero}})$  for the catchments in group 1. This model performed particularly well, and when adjusted for climate, indicated a signiﬁcant increase in. The zero ﬂow day method could be used to determine change $N_{\mathrm{zero}}$  in the occurrence of any given ﬂow in response to afforestation. The methods used in this study proved satisfactory in removing the rainfall variability, and have added useful insight into the hydrologic impacts of plantation establishment. This approach provides a methodology for understanding catchment response to afforestation, where paired catchment data is not available.  2005 Elsevier B.V. All rights reserved. $@$  Keywords: Afforestation; Flow duration curves; Flow reduction; Paired catchments



## 1. Introduction

Widespread afforestation through plantation estab- lishment on non-forested land represents a potentially signiﬁcant alteration of catchment evapotranspiration (ET). Using data collated from multiple catchment studies, researchers have demonstrated a consistent difference in ET between forests and grass or short crops, and the relationship between ET and rainfall on a mean annual basis (Holmes and Sinclair, 1986; Vertessy and Bessard, 1999; Zhang et al., 1999, 2001). Once annual rainfall exceeds  $400–500\;\mathrm{mm}$ , there is an increasing divergence between forest and grassland ET (Zhang et al., 2001). Research from South Africa in particular has demonstrated ﬂow reduction following afforestation with both pine and eucalypt species (Bosch, 1979; Van Lill et al., 1980; Van Wyk, 1987; Bosch and Von Gadow, 1990; Scott and Smith, 1997; Scott et al., 2000). In regions, where water is an increasingly valuable resource, prediction of the long-term hydrologic impact of afforestation is a prerequisite for the optimal planning of catchment land use.

Zhang et al. (1999, 2001) developed simple and easily parameterised models to predict changes in mean annual ﬂows following afforestation. However, there is a need to consider the annual ﬂow regime as the relative changes in high and low ﬂows may have considerable site speciﬁc and downstream impacts.. Sikka et al. (2003) recently showed a change from grassland to Eucalyptus globulus plantations in India decreased a low ﬂow index by a factor of two during the ﬁrst rotation (9 years), and by 3.75 during the second rotation, with more subdued impact on peak ﬂows. The index was deﬁned as the 10 day average ﬂow exceeded  $95\%$  of the time, obtained from analysis of 10-day ﬂow duration curves. Scott and Smith (1997) reported proportionally greater reductions in low ﬂows (75–100th percentiles) than annual ﬂows from South African research catchments under conversions from grass to pine and eucalypt plantations, while Bosch (1979) found the greatest reduction in seasonal ﬂow from the summer wet season. Fahey and Jackson (1997) reported the reduction in peak ﬂows was twice that of total ﬂow and low ﬂows for pine afforestation in New Zealand. The generalisations that can be drawn from annual analyses, where processes and hydrologic responses are to a certain extent integrated may not apply on a seasonal or shorter scale. Further, the observed impacts of any land use change on ﬂows may be exaggerated or understated depending on the prevailing climate. Observations of ﬂow during extended wet or dry spells, or with high annual variability can obscure the real impacts. Fig. 1 plots annual FDCsover 12 years of plantation growth for one of the catchments used in this study, Pine Creek. The net change in ﬂow is obscured by rainfall variability; e.g. the greatest change in the FDC is in 1996, with the stream ﬂowing  $<20\%$  of the time. This may be compared with 2000, where there is substantially higher ﬂows.



This paper presents the results of a project aimed at quantifying changes in annual ﬂow regime of catchments following plantation establishment. The ﬂow regime is represented by the ﬂow duration curve (FDC). The key assumption was that rainfall and forest age are the principal drivers of evapotranspira- tion. For any generalisation of response of the FDC to vegetation change, the variation in the annual climate signal must be removed. The time-tested solution to this problem is the paired-catchment (control versus treatment) experiment. The beneﬁts in such studies are manifold: unambiguous measures of trends, insights into the processes driving those trends, excellent opportunities for model parameterisation and validation. However these data are not readily available for the range of treamtments and environ- ments required. Consequently, the aims of this project were to (1) ﬁt a model to the observed annual time series of FDC percentiles; i.e. 10th percentile for each year of record with annual rainfall and plantation age as parameters, (2) replace the annual rainfall variation with the long term mean to obtain climate adjusted FDCs, and (3) quantify changes in FDC percentiles as plantations age. If the climate signal, represented by rainfall, could be successfully removed, the resulting changes in the FDC would be solely attributable to the vegetation.



## 2. Methods

## 2.1. Characterisation of ﬂow regime

Flow duration curves display the relationship between streamﬂow and the percentage of time the streamﬂow is exceeded as a cumulative density function They can be constructed for any time period (daily, weekly, monthly, etc.) and provide a graphical and statistical view of historic streamﬂow variability in a single catchment or a comparison of inter- catchment ﬂow regimes. Vogel and Fennessey (1994) and Smakhtin (1999, 2001) demonstrate the utility (and caveats) of FDCs in characterising, comparing and predicting ﬂow regimes at varying temporal scales. Fig. 1 is an example of annual FDCs constructed from daily ﬂows. For the consideration of annual ﬂow regime, daily ﬂows are an appropriate time step for FDC construction.

FDCs were computed from the distribution of daily ﬂows for each year of record based on the appropriate water years (May–April or November–October) for 10 Southern Hemisphere catchments. Each 10th percentile (decile) was extracted from the annual FDCs of each catchment to form the data sets for analysis. For the purpose of characterising changes in each of the deciles, it is assumed that the time series is principally a function of climate and vegetation characteristics. Given rainfall is generally the most important factor affecting streamﬂow and the most easily accessed data, it is chosen to represent the climate. Catchment physical properties such as soil properties and topography are assumed to be time invariant and therefore their impact on runoff is considered constant throughout the analysis. As trees intercept and transpire at increasing rates until canopy closure, a time term is required to represent plantation growth. A simple model relating the time series of each decile with rainfall and vegetation characteristics can be expressed as:



## 
$$
Q_{\%}=f(P)+g(T)
$$


(1) where  $Q_{\mathscr{V}}$  is the percentile ﬂow, is a function of $f(P)$  rainfall and  $g(T)$  is a function of the age of the plantation. Annual rainfall was chosen as the rainfall statistic as it proved to be the most robust predictor of ﬂow over the whole range of ﬂow percentiles, as compared with rainfall percentiles; e.g. median rain- fall versus 10th ﬂow percentile. The use of annual rainfall also minimises parameter complexity. The choice of model form is dependent on selecting a function that describes the relationship between forest age and ET. Scott and Smith (1997) demonstrated cumulative reductions in annual and low ﬂows resulting from afforestation ﬁtted a sigmoidal function, similar to forest growth functions. Conse- quently, we used a sigmoidal function to characterise the impact of plantation growth on each ﬂow decile. Fig. 2a is a schematic of the change in the FDC over time. The model took the form:



## 
$$
Q_{\%}=a+b(\Delta P)+\frac{Y}{1+\exp\!\left(\frac{T-T_{\mathrm{half}}}{S}\right)}
$$


(2) where  $Q_{\mathscr{V}}$  is the percentile ﬂow (i.e.  $Q_{50}$  is the 50th percentile ﬂow),  $Y$  and  $S$  are coefﬁcients of the sigmoidal term,  $\Delta P$  is the deviation of annual rainfall from the period of record average, and  $T_{\mathrm{half}}$  is the time in years at which half of the reduction in  $Q_{\mathbb{Z}}$  due to afforestation has taken place. For the average climate condition  $\Delta P\!=\!0$ ,  $a$  becomes the value of  $Q_{\%}$  when the new equilibrium plantation water use under afforestation is reached. Y then gives the magnitude of change due to afforestation, and  $S$  describes the shape of the response as shown in Fig. 2b. For the average pre-treatment condition  $\Delta P\!=\!0$  at  $T{=}0$ ,  approximately equals $Q_{\%}$   $a+Y.$ . Estimation of a pre- afforestation condition would not require the time term. Details of the optimisation scheme and sensitivity tests on initial parameter values are given in Lane et al. (2003).



## 2.2. Zero ﬂow day analysis

A notable feature of Fig. 1 is the increase in the number of zero ﬂow days. A similar approach to Eq. (2), using an inverse sigmoidal function was employed to assess the impact of afforestation on the number of zero ﬂow days per year  $(N_{\mathrm{zero}})$ . In this case, the left hand side of Eq. (2) is replaced by  $N_{\mathrm{zero}}$ , and  $b$  and  $S$  are constrained to negative as decreases as $N_{\mathrm{zero}}$  rainfall increases, and increases with plantation growth:

## 
$$
N_{\mathrm{zero}}=a+b(\Delta P)+\frac{Y}{1+\exp\!\left(\frac{T-T_{\mathrm{half}}}{S}\right)}
$$


For the average pre-treatment condition  $\Delta P\!=\!0$  and  $T\!\!=\!0,\ N_{\mathrm{zero}}$  approximately equals  $a$ .  $Y$  gives the magnitude of change in zero ﬂow days due to afforestation, and  $S$  describes the shape of the response. For the average climate condition  $\Delta P\!=\!0$ ,  becomes the number of zero ﬂow days when the $a+Y$  new equilibrium condition under afforestation is reached.



## 2.3. Statistical analyses

The coefﬁcient of efﬁciency  $(E)$  (Nash and Sutcliffe, 1970; Chiew and McMahon, 1993; Legates and McCabe, 1999) was used as the ‘goodness of ﬁt’ measure to evaluate the ﬁt between observed and predicted ﬂow deciles (2) and zero ﬂow days (3). is $E$  given by:

## 
$$
E=1.0-\frac{\sum_{i=1}^{N}(O_{i}-P_{i})^{2}}{\sum_{i=1}^{N}(O_{i}-\bar{O})^{2}}
$$


(4) where  $o$  are observed data,  $P$  are predicted values, and  $\bar{O}$  is the mean for the entire period.  $E$  is unity minus the ratio of the mean square error to the variance in the observed data, and ranges from  $-\infty$  to 1.0. Higher values indicate greater agreement between observed and predicted data as per the coefﬁcient of determination  $(r^{2})$ .  $E$  is used in preference to  $r^{2}$  in evaluating hydrologic modelling because it is a measure of the deviation from the 1:1 line. As  $E$  is always  $<r^{2}$ we have arbitrarily considered to $E\!>\!0.7$  indicate adequate model ﬁts.



It is important to assess the signiﬁcance of the model parameters to check the model assumptions that rainfall and forest age are driving changes in the FDC. The model (2) was split into simpliﬁed forms, where only the rainfall or time terms were included by setting, as shown in Eq. (5), or $b\!=\!0$   $Y{=}0$  as shown in Eq. (6). The component models (5) and (6) were then tested against the complete model, (2).

## 
$$
Q_{\%}=a+{\frac{Y}{1+\exp\!\left({\frac{T-T_{\mathrm{half}}}{S}}\right)}}
$$


(5) and



## 
$$
Q_{\%}=a+b\Delta P
$$


(6)

For both the ﬂow duration curve analysis and zero ﬂow days analysis, a  $t$ -test was then performed to test whether (5) and (6) were signiﬁcantly different to (2). A critical value of  $t$  exceeding the calculated  $t\cdot$ -value when comparing (5) and (2) would indicate the time term in (6) was required to improve the complete model and is therefore signiﬁcant, and vice versa.



Due to the constraint that the rainfall and time term must be positive, a one tailed  $t\cdot$ -test was applied. The -value was calculated as $t\mathrm{.}$   $F^{0.5}$ , and compared with the critical value for signiﬁcance at the 0.05 level. The  $F.$ -statistic was calculated as:

## 
$$
F=\mathrm{\frac{[(SSE_{s}-S S E_{c})/(d f_{c}-d f_{s})]}{S S E_{c}/d f_{c}}}
$$


(7) where SSE is the residual sum of the squared errors, df is degrees of freedom, and the subscripts s and c refer to the simpliﬁed model and complete models, respectively.



## 3. Data sets

Daily streamﬂow data were obtained from 10 catchment studies from southeastern Australia, New Zealand and South Africa. The initial criteria for selection of these catchments were a known veg- etation history and streamﬂow records of good quality. The ideal data sets were those with a lengthy pre- and post-treatment (plantation establishment) ﬂow record with approximately  $100\%$  of the catch- ment converted from grassland or a crop equivalent to plantation. In reality, all these criteria were not easy to satisfy. For example in Victoria, Australia, the best data is from Stewarts Creek, a set of decommissioned research catchments with 9 years of pre-treatment data and 25 years of post-treatment. Here, though, the treatment was a conversion from native eucalypt forest to pine. The assumption made for this data set is that the immediate post-treatment period may be viewed as a non-forested condition. This condition is likely to approximate the ET conditions of pasture or short crops for up to 3 years. Catchment details and treatments are given in Table 1.

All catchments, with the exception of Traralgon Creek, were afforested with pine species, predomi- nantly Pinus radiata, with $P.$ . patula planted at the two Cathedral Peak catchments. Traralgon Creek has only  $6\%$  pine, with the remainder eucalypts species, most of which is Eucalyptus regnans.

 indicates that the rainfall term was signiﬁcant at the  $P$  $5\%$  level,  $T$  indicates that the time term was signiﬁcant at the  $5\%$  level, \* represents signiﬁcance at the  $10\%$  level, and na denotes too few data points for meaningful analysis.

Data on soil characteristics have been obtained from published reports and personal communication with researchers, but is far from uniform, particularly regarding porosity. Consequently only an indication of mean depth is reported here. However, this does give some indication of the likely relative storage capacities of the catchments. To obtain insights into the pre-afforestation hydrologic characteristics a baseﬂow separation was performed on the daily ﬂows for the ﬁrst 3 years following disturbance, using the digital ﬁltering method of Lyne and Hollick (1979) with a ﬁlter coefﬁcient of 0.925 and three passes. The resultant average baseﬂow index (BFI), the ratio of baseﬂow to total ﬂow, is given in Table 1. The Australian catchments displaya notably lower BFI than the South African and New Zealand catchments.ForStewartsCreek,PineCreek and Redhill the lower BFI is matched by the shallow soils. Pre-treatment data is not available for all catchment in the data set, so it was decided for the sake of consistency in the analysis to start each of the data sets in the year of treatment. The FDCs were constructed for water years of May–April for eight catchments. The 2 Cathedral Peak catchments were analysed for November–October because of the summer rainfall maxima (Table 2).



## 4. Results

## 4.1. Model evaluation

The ﬁt of the complete model, Eq. (2), to the observed data was generally good. Table 2 gives

ns Indicates that no solution was found, and na denotes deciles with too few data points for analysis.

the coefﬁcient of efﬁciency  $(E)$  for each ﬂow percentile at all the catchments. The majority of ﬁts  $(77\%)$  returned  $E\!>\!0.7$ , with  $60\%~0.8$  or better. The signiﬁcance of the rainfall and time terms is given in Table 3 for all deciles, where solutions were found. There were not enough data to ﬁt the model in ﬁve instances because of extended periods of zero ﬂows. This problem is addressed to some extent in the zero ﬂow analysis. If the rainfall signal is to be separated from the vegetation signal the rainfall terms must be signiﬁcant. This term,  $b$ , was signiﬁcant for  $75\%$  of the deciles at the 0.05 level, and a further  $9\%$  at the 0.10 level. The incidence of signiﬁcance was greatest for the 10–50th percentiles at 45 of the 50 data sets at the 0.05 level. The time term,  $Y$  returned similar results, with  $80\%$  of the deciles signiﬁcant at 0.05 level. There were an additional  $9\%$  of deciles signiﬁcant at the 0.10 level.

The poorest  $E$  values were those from Lambrechts- bos A and B. The high  $E$  for 50–100th deciles at Biesievlei, where  $b$  was not signiﬁcant are notable. In general the model ﬁts the higher ﬂows (lower deciles) better, most of the poorer ﬁts are in the 80–100 percentile range. This can be expected given the results of the signiﬁcance tests for  $b$ . The results of the sensitivity analysis suggested that the  $E$  values for Glendhu 2 and for 10th and 20th percentiles from Cathedral Peak 3 may exaggerate the goodness of ﬁt to the exact form of the model (Lane et al., 2003).

## 4.2. Adjusted FDCs—magnitude of ﬂow reductions

Following the successful ﬁtting of (2) to the observed percentiles, the FDCs were adjusted for climate by setting  $\Delta P$  to zero, representing long term average annual rainfall. The climate adjusted FDCs produce an estimation of the change in ﬂow percentiles over time for each catchment due to afforestation that may be viewed in two forms: new FDCs, adjusted for climate, as exempliﬁed in Fig. 3 for Stewarts Creek 5, and a comparison between all catchments of the maximum change in yield (given by  for each ﬂow percentile from baseline ﬂows (given $Y)$  by  $a+Y)$  as shown in Fig. 4. Where the new equilibrium of maximum water use is reached, the adjusted FDCs for individual years should be identical if rainfall variability has been accounted for. The new equilibrium is approximately reached for  $T{=}2T_{\mathrm{half}}$ .

 values are given in $T_{\mathrm{half}}$  Table 4. Fig. 3 shows that for most deciles the adjusted FDCs are identical for 12 and 20 years after treatment. This ﬁgure clearly demonstrates the necessity for FDC adjustment, particularly for the 20 years FDC.

The relative net ﬂow change due to afforestation is given by  $Y/(Y+a)$ , which represents the change from the old equilibrium water use condition of pre-treatment vegetation to the new equilibrium condition at forest canopyclosure.Thisquantityisplottedforallcatchments in Fig. 4. Some deciles have been removed from the data set, the 10th and 50th percentile for Glendhu 2 and the 10th and 20th percentiles from Cathedral Peak 3. The optimised value ofwaszero or near zerofor these cases, $a$  which is not consistent with the conceptual model. The changes shown in Fig. 4 are variable. However, there are some commonalities between catchment responses. Two types of responses (groups) were identiﬁed. Group 1 catchments show a substantial increase in the number of zero ﬂow days, with a greater proportional reduction in low ﬂows than high ﬂows. Group 2 catchments show a more uniform proportional reduction in ﬂows across all percentiles, albeit with some variability. The catchments in each group are: Group 1: Stewarts Creek, Pine Creek, and Redhill Group 2: Cathedral Peak 2 and 3, Lambrechtsbos A, Lambrechtsbos B, Glendhu 2, Biesievlei and Traralgon Creek Group 1 exhibit both the highest reduction of ﬂows overall, and show the largest proportional reduction at lower ﬂows, leading to a complete cessation of ﬂow. Comparison of ﬂow reductions is hindered slightly by the range of afforestation at the catchments (Table 1). These results could be scaled up to  $100\%$  afforested if it is assumed there is a linear relationship between the area planted and ﬂow reductions. As there is no evidence that this is the case we have not presented scaled reductions here. Linear scaling would shift the reduction curves upward for those catchments that are less than  $100\%$  afforested, but would not change the shape of the curves or our groupings.

 



## 4.3. Timing of ﬂow reductions

The speed of ﬂow responses to afforestation can be evaluatedbyexaminingthevalueof( $T_{\mathrm{half}}$ Table4).There is substantial variation in response times both over the percentile spread in some individual catchments, and between the catchments. The majority of responses have a  $T_{\mathrm{half}}$  value between 5 and 10 years. Pine Creek and StewartsCreek,RedhillandLambrechtsbosAexhibitthe fastest responses, with Biesievlei showing the most uniformly slow response.  $T_{\mathrm{half}}$  for the South African catchments display a good correspondence to published annual changes (Scott et al., 2000; Van Wyk, 1987), excepting the 10–20th deciles for both Cathedral Peak

Note that no solution could be found for the 50 percentile for Glendhu indicted by the ns.

a Rainfall refers to the rainfall in the year used for comparison of results. The value in brackets refers to the deviation from the mean annual rainfall for the period of record. b Total ﬂow reduction calculated by $\textstyle\sum Y/\sum(a+Y)$  $\textstyle\sum Y/\sum(a+Y)$  for all deciles. c Low ﬂow reduction calc $a$ ated  $Y$ y P Y= PðaCYÞ d For Cathedral Peak 3 the a and Y values for the 10 and 20th percentiles were excluded as the values of  $a$  were lower then the values of the 30–100th percentiles.

catchments and the lower deciles at Lambrechtsbos B. The  $T_{\mathrm{half}}$  from Glendhu 2 appears to be substantially lower than other published data (Fahey and Jackson, 1997).

## 4.4. Comparison with paired catchment studies

A further check on the overall model performance is a comparison with published results of paired catchment studies. The data that can be compared with our results are presented in Table 5 and can be broadly compared with Fig. 4. These data are reductions in years with near average annual rainfall, and at a time after treatment when maximum changes in streamﬂow have occurred. Table 5 also includes estimates on the total and low ﬂow reductions calculated from this study. Results from Pine Creek and Traralgon Creek are not included in Table 5 as these catchments are not paired. Exact comparisons are impossible because of the rainfall variability, and lack of calibration period for Redhill. Despite this, Table 5 shows that total and low ﬂow reductions estimated from our study are comparable to the results from paired catchment studies, indicating that our simple model has successfully removed the rainfall signal.

## 4.5. Zero ﬂow days

As this analysis could only be applied, where there was consistent drying up of streams, it was conﬁned to Stewarts Creek, Pine Creek and Redhill catchments. The model returned values of  $E$  of 0.95, 0.99 and 0.99, respectively. The  $t\cdot$ -tests on  $b$  and  $\Delta N_{\mathrm{zero}}$  returned signiﬁcant results at the 0.05 level for both parameters at all three catchments. The climate adjusted zero ﬂow days are shown in Fig. 5. The increases in zero ﬂow days are substantial with ﬂows conﬁned to less than  $50\%$  of the time by year 8 at Stewarts $\mathrm{Ck}$  and Pine Ck and year 11 at Redhill. The latter has changed from an almost permanent to a highly intermittent stream. The curves are also in sensible agreement with the ﬂow reductions in Fig. 4.

## 5. Discussion

The aims of the project have largely been met. The general characterisation of FDCs and adjustment for climate has been very encouraging given the task of ﬁtting our model to 10 ﬂow percentiles, for 10 different catchments (resulting in 100 model ﬁts) with substantially varying spatial scales, soils and geology, species planted and climatic environments. Although there were poor results for individual deciles, the FDCs at eight of the 10 catchments were adequately described by Eq. (2). The results of the statistical tests in which the rainfall term was signiﬁcant for most deciles demon- strated the model structure was appropriate for adjusting the FDCs for climatic (rainfall) variability. The comparisons of our results with published paired catchment analyses are satisfactory, although the different methodologies make direct comparisons of deciles with total ﬂow uncertain. Low ﬂows at Lambrechtsbos  $\mathbf{B}$  appear to be over-estimated by our model, which is unsurprising as the model ﬁt was poor. The remaining four South African catchments, and also Redhill and Stewarts Creek are in good agreement with the published values, particularly when the deviation of average rainfall is considered. Glendhu 2 reductions are close to the reported  $27\%$ , but our model produces a heavier impact on the lower ﬂows. Overall, it appears there are no signiﬁcant discrepancies with the published paired catchment analyses. We suggest our technique represents an alternative to the paired-catchment method for assessing hydrologic response to vegetation treat- ment, where paired data are unavailable. The method has not yet resulted in a predictive model, but has increased our knowledge of afforestation impacts. This is a valuable outcome given the contentious issue of afforestation in Australia and other countries, and a current paucity of data on inter-annual ﬂows. It should be noted that nine of the 10 catchment were pine species. More data is required to compare the impact of hardwood species, particularly eucalypts, on the FDC. Unfortunately these data are currently scarce. There are substantial data on the physiological controls of eucalypt water use (see Whitehead and Beadle, 2004), but not at the catchment scale.

 



The model ﬁts show we have quantiﬁed the net impact of afforestation for the majority of the ﬂow percentiles in the 10 catchments. Results for the 10–50th percentiles were particularly encouraging. It is not surprising that the relationship between rainfall and ﬂow diminishes at lower ﬂows (60–100th percentile), where seasonal storage effects and rainfall distribution become more important drivers for runoff generation. The poorest model ﬁts were gained for Lambrechtsbos A and B. The likely reason at Lambrechtsbos A is an observed annual decrease in stand water use after 12 years (Scott et al., 2000) which does not conform to the sigmoidal form of our model over the full 19 years of record. The failure of the model to ﬁt the lower ﬂows at Lambrechtsbos  $\mathrm{B}$  is not as explicable. A decrease in stand water use in this catchment is observed as the plantation ages, but does not occur during the ﬁrst 20 years after treatment (Scott et al., 2000). Other data from South Africa (Scott et al., 2000) indicate there are diminished ﬂow reductions as plantations age, but again, generally after 20 years. Our use of an asymptotic curve assumes a new equilibrium of stand water use is reached. The results of the model ﬁtting generally justify this assumption for the length of commercial plantation growth (up to 20 years) considered here. The physio- logical relationship between stand age and water use for plantation species other than  $E.$ . regnans have not been thoroughly investigated, although Cornish and Vertessy (2001) and Roberts et al. (2001) have shown young mixed species eucalypt forests may use more water than mature stands, and Putahena and Cordery (2000) suggest maximum Pinus radiata water use may have been reached after 12 years, with a subsequent decline.



The small Australian catchments converted to pine in response group 1 (Stewarts Creek 5, Pine Creek and Redhill) have similar shallow soils, potential evapo- transpiration and rainfall distribution (relatively uni- form) although Stewarts Creek is signiﬁcantly wetter. The combination of small catchment area and the increased transpirative demand that exceeds summer and autumn rainfall and stored water results in the large impact on lower ﬂows, compared to high ﬂows.

The magnitude of the response within Group 2 varies considerably, with greater reduction in ﬂows in the two Cathedral Peak catchments, and Lambrechtsbos B. Potential evaporation is in phase with rainfall at the Cathedral Peak sites as they receive  $85\%$  ( $1260\,\mathrm{mm}$  on average) of their rainfall in summer. The conjunction of peak demand and plant water availability may explain the high reductions relative to the remaining catchments in Group 2. In addition, the stocking density was described as ‘abnormally dense’ by Scott et al. (2000). Growth at Glendhu 2 was notably slow (Fahey and Jackson, 1997) and Lambrechtsbos A and Biesievlei are described as being within sub optimal growth zones (Scott and Smith, 1997) characterised by these authors as having relatively slow response times and lesser reductions that those at more optimal sites.

The response groups may be in part explained by the storage characteristics of the catchments. Accurate measures of storage are not available from the literature, but the soil depths and the baseﬂow index (Table 1) both show the three south eastern Australian catchments with the greatest reduction are likely to have the lowest storage capacity. The greater ﬂow reductions, particu- larly for low ﬂows, could be expected under these conditions. Inclusion of a storage term in the model is an obvious option for improving the analysis. However the addition of extra parameters would be at the cost of maintaining model simplicity, particularly as character- ising a transient storage is not trivial.

Traralgon Creek would be expected to have both the most subdued ﬂow reductions and longer response time because of the large area of  $E.$ . regnans forest, and uncertain vegetation record. Peak stand water use of a natural stand of this species is around 30 years. Additionally in this large, ‘real world’ catchment, there is a continuous cycle of forest management which includes harvesting. A mixture of pasture and ‘scrub’, which could represent signiﬁcant understorey stands, were replaced by plantation species. Conse- quently the difference between pre and post treatment ET may be less than at other catchments. Reductions of this magnitude could be more readily expected in larger, multi land use catchments than the very high impacts estimated at the smaller Australian catchments.

The analysis of zero ﬂow days was successful, demonstrating that the impact on ﬂow intermittence can be evaluated without of the entire FDC. This was helpful as the change in the higher percentiles (low ﬂows) could not always be modelled. The results for the three catchments analysed are a rather stark indication of the potential for highly increased zero ﬂow periods in small catchments, at least in south-eastern Australia. However, it should be noted these curves probably represent a maximum response as they are all derived from small catchments with small storage capacities and large percentages of afforestation. This method could be used to determine change in the occurrence of any given ﬂow in response to afforestation; e.g. to determine the likelihood of maintaining a reservoir storage or an environmental ﬂow that requires an average critical ﬂow.

## 6. Summary and conclusions

Thisprojectsoughtto(i)developamethodtoremove the climate signal from streamﬂow records to identify the impact of vegetation on ﬂow from afforested catchments, and (ii) quantify this impact on the ﬂow duration curve. A simple model was proposed that considered the age of plantation and the annual rainfall to be the principal drivers for evapotranspiration. This model was ﬁtted to the observed deciles of the FDC, and the climate signal was then removed from the stream- ﬂow records by adjusting the FDC for average rainfall over the period of record. The model was tested and applied to 10 afforested catchments. We successfully ﬁtted our model to catchments with varying spatial scales, species and environments, and haveshown thatit provides a means of separating the inﬂuence of climate and vegetation on the FDCs. The modelled results showed the greatest proportional impacts were for median and lower ﬂows. The ﬂow reductions from the three small catchments SE Australian were the highest and may reﬂect lower storages. The characterisation of the number of zero ﬂow days was also successful for these catchments in indicating a signiﬁcant increase in zero ﬂows. The ﬂow reductions identiﬁed here probably represent a maximum effect given the size of the catchments, level of afforestation and the shallow soils. These results have yielded useful new insights on the contentious issue of the hydrological impact of afforestation. This research has led to the development of a method to assess the net impact of afforestation on the ﬂow duration curve which does not require paired- catchments to remove climatic variability.



## Acknowledgements

The authors would like to thank Rory Nathan, Narendra Tuteja, Tom McMahon, Geoff Podger, Rob Vertessy, Glen Walker and Peter Hairsine for particu- larly helpful discussions on methodologies and reviews, Richard Morton for valuable statistical advice, Dave Scott for supplying the South African data, Barry Fahey for the New Zealand data, and Hancocks Victorian Plantations for vegetation data. The study was funded by the Victorian Department of Natural Resources and Environment Private Forestry Unit, the CRC for Catchment Hydrology, and the MDBC funded project ‘Integrated assessment of the effects of land use changes on water yield and salt loads’ (D2013).

## References

Bosch, J.M., 1979. Treatment effects on annual and dry period streamﬂow at Cathedral Peak. South African Forestry Journal 108, 29–37.

Bosch, J.M., Von Gadow, K., 1990. Regulating afforestation for water conservation in South Africa. Suid-Afrikaanse Bosboutydskrif 153, 41–54.

Chiew, F.H.S., McMahon, T.A., 1993. Assessing the adequacy of catchment streamﬂow yield estimates. Australian Journal of Soil Research 31, 665–680.

Cornish, P.M., Vertessy, R.A., 2001. Forest age-induced changes in evapotranspiration and water yield in a eucalypt forest. Journal of Hydrology 242, 43–63.

Fahey, B., Jackson, R., 1997. Hydrological impacts of converting nativeforestsandgrasslandstopineplantations,South Island, New Zealand. Agricultural and Forest Meteorology 84, 69–82.

Hickel, K., 2001. The effect of pine afforestation on ﬂow regime in small upland catchments. Masters Thesis, University of Stuttgart, p. 134.

Holmes, J.W., Sinclair, J.A., 1986. Water yield from some afforested catchments in Victoria. In Hydrology and Water Resources Symposium, Grifﬁth University, Brisbane 25–27 November 1986, pp. 214–218.

Lane, P.N.J., Best, A.E., Hickel, K., Zhang, L., 2003. The effect of afforestation on ﬂow duration curves. Cooperative Research Centre for Catchment Hydrology Technical Report 03/13, p. 25.

Legates, D.R., McCabe, G.J., 1999. Evaluating the use of ‘goodness- of-ﬁt’ measures in hydrologic and hydroclimatic model validation. Water Resources Research 35, 233–241.

Lyne, V.D., Hollick, M., 1979. Stochastic time-varying rainfall–runoff modelling. Hydrology and Water Resources Symposium, Perth. Institution of Engineers, Australia, pp. 89–92.

Nandakumar, N., Mein, R.G., 1993. Analysis of paired catchment data to determine the hydrologic effects of changes in vegetative cover on yield. Technical Report for Project UM010, Monash University Dept. of Civil Engineering, 219 p.

Nash, J.E., Sutcliffe, J.V., 1970. River ﬂow forecasting through conceptual models, I, A discussion of principals. Journal of Hydrology 10, 282–290.

Putahena, W.M., Cordery, I., 2000. Some hydrological effects of changing forest cover from eucalyptus to Pinus radiata. Agricul- tural and Forest Meteorology 100, 59–72.

Roberts, S., Vertessy, R.A., Grayson, R.G., 2001. Transpiration from Eucalyptus sieberi (L. Johnson) forests of different age. Forest Ecology and Management 143, 153–161.

Scott, D.F., Smith, R.E., 1997. Preliminary empirical models to predict reductions in total and low ﬂows resulting from afforestation. Water S.A. 23, 135–140.

Scott,D.F.,Prinsloo,F.W.,Moses,G.,Mehlomakulu,M., Simmers, A.D.A., 2000. Area-analysis of the South African catchmentafforestationexperimentaldata.WRCReport No. 810/1/00.

Sikka, A.K., Samra, J.S., Sharda, V.N., Samraj, P., Lakshmanan, V., 2003. Low ﬂow and high responses to converting natural grassland into bluegum (Eucalyptus globulus) in Ningiris watersheds of South India. Journal of Hydrology 270, 12–26.

Smakhtin, V.U., 1999. A concept of pragmatic hydrological time series modelling and its application in South African context. In Ninth South African National Hydrology Symposium, 29–30 November 1999, pp. 1–11.

Smakhtin, V.U., 2001. Low ﬂow hydrology: a review. Journal of Hydrology 240, 147–186.

Van Lill, W.S., Kruger, F.J., Van Wyk, D.B., 1980. The effect of afforestation with Eucalyptus grandis Hill ex Maiden and Pinus patula Schlect. et Cham. on streamﬂow from experimental catchments at Mokubulaan, Transvaal. Journal of Hydrology 48, 107–118.



Van Wyk, D.B., 1987. Some effects of afforestation on streamﬂow in the Western Cape Province, South Africa. Water S.A. 13, 31–36.

Vertessy, R.A., Bessard, Y., 1999. Anticipating the negative hydrologic effects of plantation expansion: results from a GIS-based analysis on the Murrumbidgee Basin, in: Croke, J., Lane, P.N.J. (Eds.), Forest Management for Water Quality and Quantity: Proceedings of the 2nd Erosion in Forests Meeting Cooperative Research Centre for Catchment Hydrology, Report 99/6, pp. 69–73.

Vogel, R.M., Fennessey, N.M., 1994. Flow duration curves. 1. New interpretation and conﬁdence intervals. Journal of Water Planning and Management 120 (4), 485–504.

Whitehead, D., Beadle C.L., 2004. Physiological regulation of productivity and water use in Eucalyptus: a review. Forest Ecology and Management, 193, 113–140.

Zhang, L., Dawes, W.R., Walker, G.R., 1999. Predicting the effect of vegetation changes on catchment average water balance. Coop- erative Research Centre for Catchment Hydrology Technical Report 99/12, p. 35.

Zhang, L., Dawes, W.R., Walker, G.R., 2001. Response of mean annual evapotranspiration to vegetation changes at catchment scale. Water Resources Research 37, 701–708.
organic chemistry, review Z 0200

51 - 363

1
organic chemistry, review Z 0200

51 - 363

1


Short Communication

# Detection of Neorickettsia risticii from various freshwater snail species collected from a district irrigation canal in Nevada County, California

Nicola Pusterla a,⇑, Daniel Hagerty , Samantha Mapes , Josh VanGeem , Lindsey T. Groves b, Mario Dinucci , Langdon C. Fielding , Jill C. Higgins d

 Department of Medicine and Epidemiology, School of Veterinary Medicine, University of California, Davis, CA 95616, USA  Natural History Museum of Los Angeles County, Los Angeles, CA 90007, USA  Sierra Equine, Grass Valley, CA 95945, USA d Loomis Basin Equine Medical Center, Loomis, CA 95650, USA

## a r t i c l ei n f o

## a b s t r a c t

Article history: Accepted 26 February 2013 Keywords: Neorickettsia risticii Equine neorickettsiosis Horse Freshwater snails Irrigation canal Potomac horse fever This study investigated the role of a district irrigation canal in Nevada County, California, USA, as the point source of infection for Neorickettsia risticii, causative agent of equine neorickettsiosis (EN). A total of 568 freshwater snails comprising spp., (Carpenter, 1857) (Rough Rams- horn), (Gould, 1855) (Protean Physa) and feces from three horses with EN were collected and tested for  $N.$  risticii by real-time PCR. A total of four freshwater snails tested PCR positive for N. risticii. Phylogenetic analysis showed  $99.8–100\%$  homology between the different snail and horse N. risticii iso- lates. This study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and sug- gests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

 



2013 Elsevier Ltd. All rights reserved. $@$  Neorickettsia risticii is the etiologic agent of equine neoricketts- iosis (EN), or Potomac horse fever (PHF), an acute enterocolitis of horses. has recently been associated with trem- atode-vectors, which use freshwater snails and aquatic insects as intermediate hosts and insectivorous birds and bats as deﬁnitive hosts (Pusterla et al., 2003; Gibson et al., 2005). In Northern Cali- fornia, the species of snail incriminated in the life cycle of the hel- minth vector infected with  $N.$  risticii has been identiﬁed as Juga yrekaensis (Henderson, 1935, [Yreka juga]), a common pleurocerid snail, which inhabits fresh or brackish stream water in the North- western United States (Barlough et al., 1998). Additionally, DNA from N. risticii has been detected in virgulate cercariae in lymnaeid snails (Stagnicola spp.) from Northern California, in virgulate xiph- idiocercariae isolated from pleurocerid snails (Elimia livescens, (Menke, 1830), [Liver Elimia]) in central Ohio, and from pleuroce- rid snails (Elimia virginica, (Say, 1817), [Piedmont Elimia]) in cen- tral Pennsylvania, indicating that other species of freshwater snails may also harbor infected trematodes (Reubel et al., 1998; Kanter et al., 2000).



In recent years, there have been an increasing number of re- ported EN cases from Nevada County, California. Of epidemiologi- cal interest is the fact that water is supplied to the premises where conﬁrmed EN cases have historically been diagnosed through a dis- trict irrigation canal system. The aim of this study was, therefore, to determine if freshwater snails collected from these canals har- bored N. risticii.



A total of 568 freshwater snails, including 218 Juga spp., 140 Planorbella subcrenata and 210 Physella virgata (Fig. 1), were collected from ﬁve different sites along the irrigation canal from September 2010 to August 2011 (Fig. 2). Each snail was extracted from its shell using clean forceps and scalpel blades, and placed into a 96-well plate containing  $500\,\upmu\mathrm{L}$ of lysis solution. All extrac- tion steps were performed in a hood with laminar ﬂow. Proteinase K and two grinding beads ( $\mathrm{\Omega}.4\,\mathrm{mm}$  diameter, stainless steel beads, SpexCertiprep) were added and the tissues homogenized in a GenoGrinder2000 (SpexCertiprep) for 2 min at 1000 strokes per minute.

Protein digest was at  $56\,^{\circ}C$ for  $30\,\mathrm{min}$ followed by a  $30\,\mathrm{min}$ per- iod at  $-20\,^{\circ}\mathrm{C}$ to reduce foam. Two hundred microliters of the mix- ture were processed for DNA puriﬁcation using an automated extraction protocol as previously reported (Pusterla et al., 2000a). Extracted nucleic acids were tested for the presence of  $N.$  risticii by real-time PCR using a previously established and validated pro- tocol (Pusterla et al., 2000a). Known positive and negative controls were used with every PCR run. Furthermore, a real-time PCR assay targeting the universal eukaryotic 18S rRNA gene was used as quality control (i.e. efﬁciency of DNA puriﬁcation and ampliﬁca- tion) and as an indicator of inhibition (Eukaryotic 18S rRNA Endog- enous Control, Applied Biosystems).

Three horses diagnosed with PHF from different localities in Ne- vada County were used for phylogenetic analysis. A 527 nucleotide segment of the 16S rRNA gene of N. risticii was generated from the snail and horse isolates in order to determine phylogenetic rela- tionship (Reubel et al., 1998).

Nucleic acid extraction was successful in all 568 freshwater snails based on the detection of the 18S rRNA gene. Four freshwa- ter snails tested PCR positive for N. risticii. These included one Juga species (0.4%) and three P. subcrenata (2.1%) snails, but none of the P. virgata snails tested PCR positive for N. risticii. All three horses presented to Loomis Basin Equine Medical Center with classical signs of EN, including depression, anorexia, fever, colic and soft- formed feces tested PCR positive for N. risticii in feces (Bertin et al., 2011). Sequence comparison of a 527 nucleotide segment of the 16S rRNA gene of N. risticii showed 100% homology amongst the snails and between 99.3% and 100.0% homology with the three horses from Nevada County (Fig. 3).

The biology of N. risticii is associated with an aquatic environ- ment, which explains why EN cases have a seasonal occurrence and are commonly observed near waterways in endemic areas (Barlough et al., 1998). This study reports on the detection of N. ris- ticii in freshwater snails and horses from Nevada County, Califor- nia.Thisnewlyrecognizedendemicareaisgeographically distinct from other endemic areas in California, which are located close to the Oregon border. In the rural area of Nevada County water is provided via a canal system from April to October and used mainly for local agriculture, including pasture irrigation. The water originates from Scotts Flat Reservoir and is carried in an open canal system for over 150 miles (ca. 241 km) to approxi- mately 32,000 irrigable acres (ca. 13,000 ha). To the knowledge of the authors, this is the ﬁrst report associating a district irrigation canal system with the spread of N. risticii.

Demonstrating endemic occurrence of N. risticii was based on the detection of the neorickettsial organism in freshwater snails and resident horses with signs compatible with EN. Previous work has shown that the number of PCR positive snails appears to de- pend on the size of the snails, the month of collection, and geo- graphic origin (Pusterla et al., 2000b). In the present study, we detected a prevalence of infected freshwater snails ranging from  $0.4\%$  to  $2.1\%$  which is in agreement with previous reports (Barlough et al., 1998; Pusterla et al., 2000b). is considered the main species of freshwater snails associated with the life cycle of the digenetic trematode vectors of  $N.$  risticii in Northern California (Reubel et al., 1998). This was in contrast with the present study, where the majority of the PCR positive freshwater snails belonged to P. subcrenata. This planorbid snail is common in the Northwest- ern United States and can be recognized by its sinistrally-coiled shell (Burch, 1989). Previous work has shown that in endemic areas various freshwater snail species can harbor (Reubel et al., 1998).

EN was diagnosed in the horses from Nevada County based on clinical signs, laboratory detection of  $N.$  risticii via PCR and re- sponse to treatment with oxytetracycline and supportive care as previously reported (Bertin et al., 2011). All the affected horses had direct access to irrigated pastures and lived  $100{-}300\,\mathrm{m}$  from the nearest irrigation canal. The close relationship between fresh- water snail and horse  $N.$  risticii isolates was further supported by the high sequence homology of the partial sequence of the 16S rRNA gene of  $N.$  risticii (Reubel et al., 1998). The study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and suggests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

### Conﬂict of interest statement

### Acknowledgement

The authors wish to thank Edward J. Johannes (Deixis Consul- tants, SeaTac, WA) for conﬁrming the identiﬁcation of the Juga specimens.

### References

Barlough, J.E., Reubel, G.H., Madigan, J.E., Vredevoe, L.K., Miller, P.E., Rikihisa, Y., 1998. Detection of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Pleuroceridae: Juga spp.) from northern California. Applied and Environmental Microbiology 64, 2888–2893.

Bertin, F.R., Reising, A., Slovis, N.M., Constable, P.D., Taylor, S.D., 2011. Factors associated with survival in 50 horses with Potomac horse fever. In: Proceedings of the Annual Meeting of the American College of Veterinary Internal Medicine, New Orleans, USA, pp. 263.

Burch, J.B., 1989. North American Freshwater Snails. Malocological Publications, Hamburg, MI, USA, pp. 365.

Gibson, K.E., Rikihisa, Y., Zhang, C., Martin, C., 2005. is vertically transmittedinthe trematodeAcanthatriumoregonenseand horizontally transmitted to bats. Environmental Microbiology 7, 203–212.

Kanter, M., Mott, J., Ohashi, N., Fried, B., Reed, S., Lin, Y.C., Rikihisa, Y., 2000. Analysis of 16S rRNA and 51-kilodalton antigen gene and transmission in mice of Ehrlichia risticii in virgulate trematodes from Elimia livescens snails in Ohio. Journal of Clinical Microbiology 38, 3349–3358.

Pusterla, N., Leutenegger, C.M., Sigrist, B., Chae, J.S., Lutz, H., Madigan, J.E., 2000a. Detection and quantitation of Ehrlichia risticii genomic DNA in infected horses and snails by real-time PCR. Veterinary Parasitology 90, 129–135.

Pusterla, N., Johnson, E., Chae, J., Pusterla, J.B., DeRock, E., Madigan, J.E., 2000b. Infection rate of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Juga yrekaensis) from northern California. Veterinary Parasitology 92, 151–156.

Pusterla, N., Johnson, E.M., Chae, J.S., Madigan, J.E., 2003. Digenetic trematodes, Acanthatrium sp. and Lecithodendrium sp., as vectors of Neorickettsia risticii, the agent of Potomac horse fever. Journal of Helminthology 77, 335–339.

Reubel, G.H., Barlough, J.E., Madigan, J.E., 1998. Production and characterization of Ehrlichia risticii, the agent of Potomac horse fever, from snails (Pleuroceridae: Juga spp.) in aquarium culture and genetic comparison to equine strains. Journal of Clinical Microbiology 36, 1501–1511.

The authors have no ﬁnancial or personal relationship with other people or organizations that could inappropriately inﬂuence or bias the content of the paper.
Short Communication

# Detection of Neorickettsia risticii from various freshwater snail species collected from a district irrigation canal in Nevada County, California

Nicola Pusterla a,⇑, Daniel Hagerty , Samantha Mapes , Josh VanGeem , Lindsey T. Groves b, Mario Dinucci , Langdon C. Fielding , Jill C. Higgins d

 Department of Medicine and Epidemiology, School of Veterinary Medicine, University of California, Davis, CA 95616, USA  Natural History Museum of Los Angeles County, Los Angeles, CA 90007, USA  Sierra Equine, Grass Valley, CA 95945, USA d Loomis Basin Equine Medical Center, Loomis, CA 95650, USA

## a r t i c l ei n f o

## a b s t r a c t

Article history: Accepted 26 February 2013 Keywords: Neorickettsia risticii Equine neorickettsiosis Horse Freshwater snails Irrigation canal Potomac horse fever This study investigated the role of a district irrigation canal in Nevada County, California, USA, as the point source of infection for Neorickettsia risticii, causative agent of equine neorickettsiosis (EN). A total of 568 freshwater snails comprising spp., (Carpenter, 1857) (Rough Rams- horn), (Gould, 1855) (Protean Physa) and feces from three horses with EN were collected and tested for  $N.$  risticii by real-time PCR. A total of four freshwater snails tested PCR positive for N. risticii. Phylogenetic analysis showed  $99.8–100\%$  homology between the different snail and horse N. risticii iso- lates. This study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and sug- gests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

 



2013 Elsevier Ltd. All rights reserved. $@$  Neorickettsia risticii is the etiologic agent of equine neoricketts- iosis (EN), or Potomac horse fever (PHF), an acute enterocolitis of horses. has recently been associated with trem- atode-vectors, which use freshwater snails and aquatic insects as intermediate hosts and insectivorous birds and bats as deﬁnitive hosts (Pusterla et al., 2003; Gibson et al., 2005). In Northern Cali- fornia, the species of snail incriminated in the life cycle of the hel- minth vector infected with  $N.$  risticii has been identiﬁed as Juga yrekaensis (Henderson, 1935, [Yreka juga]), a common pleurocerid snail, which inhabits fresh or brackish stream water in the North- western United States (Barlough et al., 1998). Additionally, DNA from N. risticii has been detected in virgulate cercariae in lymnaeid snails (Stagnicola spp.) from Northern California, in virgulate xiph- idiocercariae isolated from pleurocerid snails (Elimia livescens, (Menke, 1830), [Liver Elimia]) in central Ohio, and from pleuroce- rid snails (Elimia virginica, (Say, 1817), [Piedmont Elimia]) in cen- tral Pennsylvania, indicating that other species of freshwater snails may also harbor infected trematodes (Reubel et al., 1998; Kanter et al., 2000).



In recent years, there have been an increasing number of re- ported EN cases from Nevada County, California. Of epidemiologi- cal interest is the fact that water is supplied to the premises where conﬁrmed EN cases have historically been diagnosed through a dis- trict irrigation canal system. The aim of this study was, therefore, to determine if freshwater snails collected from these canals har- bored N. risticii.



A total of 568 freshwater snails, including 218 Juga spp., 140 Planorbella subcrenata and 210 Physella virgata (Fig. 1), were collected from ﬁve different sites along the irrigation canal from September 2010 to August 2011 (Fig. 2). Each snail was extracted from its shell using clean forceps and scalpel blades, and placed into a 96-well plate containing  $500\,\upmu\mathrm{L}$ of lysis solution. All extrac- tion steps were performed in a hood with laminar ﬂow. Proteinase K and two grinding beads ( $\mathrm{\Omega}.4\,\mathrm{mm}$  diameter, stainless steel beads, SpexCertiprep) were added and the tissues homogenized in a GenoGrinder2000 (SpexCertiprep) for 2 min at 1000 strokes per minute.

Protein digest was at  $56\,^{\circ}C$ for  $30\,\mathrm{min}$ followed by a  $30\,\mathrm{min}$ per- iod at  $-20\,^{\circ}\mathrm{C}$ to reduce foam. Two hundred microliters of the mix- ture were processed for DNA puriﬁcation using an automated extraction protocol as previously reported (Pusterla et al., 2000a). Extracted nucleic acids were tested for the presence of  $N.$  risticii by real-time PCR using a previously established and validated pro- tocol (Pusterla et al., 2000a). Known positive and negative controls were used with every PCR run. Furthermore, a real-time PCR assay targeting the universal eukaryotic 18S rRNA gene was used as quality control (i.e. efﬁciency of DNA puriﬁcation and ampliﬁca- tion) and as an indicator of inhibition (Eukaryotic 18S rRNA Endog- enous Control, Applied Biosystems).

Three horses diagnosed with PHF from different localities in Ne- vada County were used for phylogenetic analysis. A 527 nucleotide segment of the 16S rRNA gene of N. risticii was generated from the snail and horse isolates in order to determine phylogenetic rela- tionship (Reubel et al., 1998).

Nucleic acid extraction was successful in all 568 freshwater snails based on the detection of the 18S rRNA gene. Four freshwa- ter snails tested PCR positive for N. risticii. These included one Juga species (0.4%) and three P. subcrenata (2.1%) snails, but none of the P. virgata snails tested PCR positive for N. risticii. All three horses presented to Loomis Basin Equine Medical Center with classical signs of EN, including depression, anorexia, fever, colic and soft- formed feces tested PCR positive for N. risticii in feces (Bertin et al., 2011). Sequence comparison of a 527 nucleotide segment of the 16S rRNA gene of N. risticii showed 100% homology amongst the snails and between 99.3% and 100.0% homology with the three horses from Nevada County (Fig. 3).

The biology of N. risticii is associated with an aquatic environ- ment, which explains why EN cases have a seasonal occurrence and are commonly observed near waterways in endemic areas (Barlough et al., 1998). This study reports on the detection of N. ris- ticii in freshwater snails and horses from Nevada County, Califor- nia.Thisnewlyrecognizedendemicareaisgeographically distinct from other endemic areas in California, which are located close to the Oregon border. In the rural area of Nevada County water is provided via a canal system from April to October and used mainly for local agriculture, including pasture irrigation. The water originates from Scotts Flat Reservoir and is carried in an open canal system for over 150 miles (ca. 241 km) to approxi- mately 32,000 irrigable acres (ca. 13,000 ha). To the knowledge of the authors, this is the ﬁrst report associating a district irrigation canal system with the spread of N. risticii.

Demonstrating endemic occurrence of N. risticii was based on the detection of the neorickettsial organism in freshwater snails and resident horses with signs compatible with EN. Previous work has shown that the number of PCR positive snails appears to de- pend on the size of the snails, the month of collection, and geo- graphic origin (Pusterla et al., 2000b). In the present study, we detected a prevalence of infected freshwater snails ranging from  $0.4\%$  to  $2.1\%$  which is in agreement with previous reports (Barlough et al., 1998; Pusterla et al., 2000b). is considered the main species of freshwater snails associated with the life cycle of the digenetic trematode vectors of  $N.$  risticii in Northern California (Reubel et al., 1998). This was in contrast with the present study, where the majority of the PCR positive freshwater snails belonged to P. subcrenata. This planorbid snail is common in the Northwest- ern United States and can be recognized by its sinistrally-coiled shell (Burch, 1989). Previous work has shown that in endemic areas various freshwater snail species can harbor (Reubel et al., 1998).

EN was diagnosed in the horses from Nevada County based on clinical signs, laboratory detection of  $N.$  risticii via PCR and re- sponse to treatment with oxytetracycline and supportive care as previously reported (Bertin et al., 2011). All the affected horses had direct access to irrigated pastures and lived  $100{-}300\,\mathrm{m}$  from the nearest irrigation canal. The close relationship between fresh- water snail and horse  $N.$  risticii isolates was further supported by the high sequence homology of the partial sequence of the 16S rRNA gene of  $N.$  risticii (Reubel et al., 1998). The study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and suggests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

### Conﬂict of interest statement

### Acknowledgement

The authors wish to thank Edward J. Johannes (Deixis Consul- tants, SeaTac, WA) for conﬁrming the identiﬁcation of the Juga specimens.

### References

Barlough, J.E., Reubel, G.H., Madigan, J.E., Vredevoe, L.K., Miller, P.E., Rikihisa, Y., 1998. Detection of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Pleuroceridae: Juga spp.) from northern California. Applied and Environmental Microbiology 64, 2888–2893.

Bertin, F.R., Reising, A., Slovis, N.M., Constable, P.D., Taylor, S.D., 2011. Factors associated with survival in 50 horses with Potomac horse fever. In: Proceedings of the Annual Meeting of the American College of Veterinary Internal Medicine, New Orleans, USA, pp. 263.

Burch, J.B., 1989. North American Freshwater Snails. Malocological Publications, Hamburg, MI, USA, pp. 365.

Gibson, K.E., Rikihisa, Y., Zhang, C., Martin, C., 2005. is vertically transmittedinthe trematodeAcanthatriumoregonenseand horizontally transmitted to bats. Environmental Microbiology 7, 203–212.

Kanter, M., Mott, J., Ohashi, N., Fried, B., Reed, S., Lin, Y.C., Rikihisa, Y., 2000. Analysis of 16S rRNA and 51-kilodalton antigen gene and transmission in mice of Ehrlichia risticii in virgulate trematodes from Elimia livescens snails in Ohio. Journal of Clinical Microbiology 38, 3349–3358.

Pusterla, N., Leutenegger, C.M., Sigrist, B., Chae, J.S., Lutz, H., Madigan, J.E., 2000a. Detection and quantitation of Ehrlichia risticii genomic DNA in infected horses and snails by real-time PCR. Veterinary Parasitology 90, 129–135.

Pusterla, N., Johnson, E., Chae, J., Pusterla, J.B., DeRock, E., Madigan, J.E., 2000b. Infection rate of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Juga yrekaensis) from northern California. Veterinary Parasitology 92, 151–156.

Pusterla, N., Johnson, E.M., Chae, J.S., Madigan, J.E., 2003. Digenetic trematodes, Acanthatrium sp. and Lecithodendrium sp., as vectors of Neorickettsia risticii, the agent of Potomac horse fever. Journal of Helminthology 77, 335–339.

Reubel, G.H., Barlough, J.E., Madigan, J.E., 1998. Production and characterization of Ehrlichia risticii, the agent of Potomac horse fever, from snails (Pleuroceridae: Juga spp.) in aquarium culture and genetic comparison to equine strains. Journal of Clinical Microbiology 36, 1501–1511.

The authors have no ﬁnancial or personal relationship with other people or organizations that could inappropriately inﬂuence or bias the content of the paper.
Short Communication

# Detection of Neorickettsia risticii from various freshwater snail species collected from a district irrigation canal in Nevada County, California

Nicola Pusterla a,⇑, Daniel Hagerty , Samantha Mapes , Josh VanGeem , Lindsey T. Groves b, Mario Dinucci , Langdon C. Fielding , Jill C. Higgins d

 Department of Medicine and Epidemiology, School of Veterinary Medicine, University of California, Davis, CA 95616, USA  Natural History Museum of Los Angeles County, Los Angeles, CA 90007, USA  Sierra Equine, Grass Valley, CA 95945, USA d Loomis Basin Equine Medical Center, Loomis, CA 95650, USA

## a r t i c l ei n f o

## a b s t r a c t

Article history: Accepted 26 February 2013 Keywords: Neorickettsia risticii Equine neorickettsiosis Horse Freshwater snails Irrigation canal Potomac horse fever This study investigated the role of a district irrigation canal in Nevada County, California, USA, as the point source of infection for Neorickettsia risticii, causative agent of equine neorickettsiosis (EN). A total of 568 freshwater snails comprising spp., (Carpenter, 1857) (Rough Rams- horn), (Gould, 1855) (Protean Physa) and feces from three horses with EN were collected and tested for  $N.$  risticii by real-time PCR. A total of four freshwater snails tested PCR positive for N. risticii. Phylogenetic analysis showed  $99.8–100\%$  homology between the different snail and horse N. risticii iso- lates. This study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and sug- gests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

 



2013 Elsevier Ltd. All rights reserved. $@$  Neorickettsia risticii is the etiologic agent of equine neoricketts- iosis (EN), or Potomac horse fever (PHF), an acute enterocolitis of horses. has recently been associated with trem- atode-vectors, which use freshwater snails and aquatic insects as intermediate hosts and insectivorous birds and bats as deﬁnitive hosts (Pusterla et al., 2003; Gibson et al., 2005). In Northern Cali- fornia, the species of snail incriminated in the life cycle of the hel- minth vector infected with  $N.$  risticii has been identiﬁed as Juga yrekaensis (Henderson, 1935, [Yreka juga]), a common pleurocerid snail, which inhabits fresh or brackish stream water in the North- western United States (Barlough et al., 1998). Additionally, DNA from N. risticii has been detected in virgulate cercariae in lymnaeid snails (Stagnicola spp.) from Northern California, in virgulate xiph- idiocercariae isolated from pleurocerid snails (Elimia livescens, (Menke, 1830), [Liver Elimia]) in central Ohio, and from pleuroce- rid snails (Elimia virginica, (Say, 1817), [Piedmont Elimia]) in cen- tral Pennsylvania, indicating that other species of freshwater snails may also harbor infected trematodes (Reubel et al., 1998; Kanter et al., 2000).



In recent years, there have been an increasing number of re- ported EN cases from Nevada County, California. Of epidemiologi- cal interest is the fact that water is supplied to the premises where conﬁrmed EN cases have historically been diagnosed through a dis- trict irrigation canal system. The aim of this study was, therefore, to determine if freshwater snails collected from these canals har- bored N. risticii.



A total of 568 freshwater snails, including 218 Juga spp., 140 Planorbella subcrenata and 210 Physella virgata (Fig. 1), were collected from ﬁve different sites along the irrigation canal from September 2010 to August 2011 (Fig. 2). Each snail was extracted from its shell using clean forceps and scalpel blades, and placed into a 96-well plate containing  $500\,\upmu\mathrm{L}$ of lysis solution. All extrac- tion steps were performed in a hood with laminar ﬂow. Proteinase K and two grinding beads ( $\mathrm{\Omega}.4\,\mathrm{mm}$  diameter, stainless steel beads, SpexCertiprep) were added and the tissues homogenized in a GenoGrinder2000 (SpexCertiprep) for 2 min at 1000 strokes per minute.

Protein digest was at  $56\,^{\circ}C$ for  $30\,\mathrm{min}$ followed by a  $30\,\mathrm{min}$ per- iod at  $-20\,^{\circ}\mathrm{C}$ to reduce foam. Two hundred microliters of the mix- ture were processed for DNA puriﬁcation using an automated extraction protocol as previously reported (Pusterla et al., 2000a). Extracted nucleic acids were tested for the presence of  $N.$  risticii by real-time PCR using a previously established and validated pro- tocol (Pusterla et al., 2000a). Known positive and negative controls were used with every PCR run. Furthermore, a real-time PCR assay targeting the universal eukaryotic 18S rRNA gene was used as quality control (i.e. efﬁciency of DNA puriﬁcation and ampliﬁca- tion) and as an indicator of inhibition (Eukaryotic 18S rRNA Endog- enous Control, Applied Biosystems).

Three horses diagnosed with PHF from different localities in Ne- vada County were used for phylogenetic analysis. A 527 nucleotide segment of the 16S rRNA gene of N. risticii was generated from the snail and horse isolates in order to determine phylogenetic rela- tionship (Reubel et al., 1998).

Nucleic acid extraction was successful in all 568 freshwater snails based on the detection of the 18S rRNA gene. Four freshwa- ter snails tested PCR positive for N. risticii. These included one Juga species (0.4%) and three P. subcrenata (2.1%) snails, but none of the P. virgata snails tested PCR positive for N. risticii. All three horses presented to Loomis Basin Equine Medical Center with classical signs of EN, including depression, anorexia, fever, colic and soft- formed feces tested PCR positive for N. risticii in feces (Bertin et al., 2011). Sequence comparison of a 527 nucleotide segment of the 16S rRNA gene of N. risticii showed 100% homology amongst the snails and between 99.3% and 100.0% homology with the three horses from Nevada County (Fig. 3).

The biology of N. risticii is associated with an aquatic environ- ment, which explains why EN cases have a seasonal occurrence and are commonly observed near waterways in endemic areas (Barlough et al., 1998). This study reports on the detection of N. ris- ticii in freshwater snails and horses from Nevada County, Califor- nia.Thisnewlyrecognizedendemicareaisgeographically distinct from other endemic areas in California, which are located close to the Oregon border. In the rural area of Nevada County water is provided via a canal system from April to October and used mainly for local agriculture, including pasture irrigation. The water originates from Scotts Flat Reservoir and is carried in an open canal system for over 150 miles (ca. 241 km) to approxi- mately 32,000 irrigable acres (ca. 13,000 ha). To the knowledge of the authors, this is the ﬁrst report associating a district irrigation canal system with the spread of N. risticii.

Demonstrating endemic occurrence of N. risticii was based on the detection of the neorickettsial organism in freshwater snails and resident horses with signs compatible with EN. Previous work has shown that the number of PCR positive snails appears to de- pend on the size of the snails, the month of collection, and geo- graphic origin (Pusterla et al., 2000b). In the present study, we detected a prevalence of infected freshwater snails ranging from  $0.4\%$  to  $2.1\%$  which is in agreement with previous reports (Barlough et al., 1998; Pusterla et al., 2000b). is considered the main species of freshwater snails associated with the life cycle of the digenetic trematode vectors of  $N.$  risticii in Northern California (Reubel et al., 1998). This was in contrast with the present study, where the majority of the PCR positive freshwater snails belonged to P. subcrenata. This planorbid snail is common in the Northwest- ern United States and can be recognized by its sinistrally-coiled shell (Burch, 1989). Previous work has shown that in endemic areas various freshwater snail species can harbor (Reubel et al., 1998).

EN was diagnosed in the horses from Nevada County based on clinical signs, laboratory detection of  $N.$  risticii via PCR and re- sponse to treatment with oxytetracycline and supportive care as previously reported (Bertin et al., 2011). All the affected horses had direct access to irrigated pastures and lived  $100{-}300\,\mathrm{m}$  from the nearest irrigation canal. The close relationship between fresh- water snail and horse  $N.$  risticii isolates was further supported by the high sequence homology of the partial sequence of the 16S rRNA gene of  $N.$  risticii (Reubel et al., 1998). The study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and suggests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

### Conﬂict of interest statement

### Acknowledgement

The authors wish to thank Edward J. Johannes (Deixis Consul- tants, SeaTac, WA) for conﬁrming the identiﬁcation of the Juga specimens.

### References

Barlough, J.E., Reubel, G.H., Madigan, J.E., Vredevoe, L.K., Miller, P.E., Rikihisa, Y., 1998. Detection of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Pleuroceridae: Juga spp.) from northern California. Applied and Environmental Microbiology 64, 2888–2893.

Bertin, F.R., Reising, A., Slovis, N.M., Constable, P.D., Taylor, S.D., 2011. Factors associated with survival in 50 horses with Potomac horse fever. In: Proceedings of the Annual Meeting of the American College of Veterinary Internal Medicine, New Orleans, USA, pp. 263.

Burch, J.B., 1989. North American Freshwater Snails. Malocological Publications, Hamburg, MI, USA, pp. 365.

Gibson, K.E., Rikihisa, Y., Zhang, C., Martin, C., 2005. is vertically transmittedinthe trematodeAcanthatriumoregonenseand horizontally transmitted to bats. Environmental Microbiology 7, 203–212.

Kanter, M., Mott, J., Ohashi, N., Fried, B., Reed, S., Lin, Y.C., Rikihisa, Y., 2000. Analysis of 16S rRNA and 51-kilodalton antigen gene and transmission in mice of Ehrlichia risticii in virgulate trematodes from Elimia livescens snails in Ohio. Journal of Clinical Microbiology 38, 3349–3358.

Pusterla, N., Leutenegger, C.M., Sigrist, B., Chae, J.S., Lutz, H., Madigan, J.E., 2000a. Detection and quantitation of Ehrlichia risticii genomic DNA in infected horses and snails by real-time PCR. Veterinary Parasitology 90, 129–135.

Pusterla, N., Johnson, E., Chae, J., Pusterla, J.B., DeRock, E., Madigan, J.E., 2000b. Infection rate of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Juga yrekaensis) from northern California. Veterinary Parasitology 92, 151–156.

Pusterla, N., Johnson, E.M., Chae, J.S., Madigan, J.E., 2003. Digenetic trematodes, Acanthatrium sp. and Lecithodendrium sp., as vectors of Neorickettsia risticii, the agent of Potomac horse fever. Journal of Helminthology 77, 335–339.

Reubel, G.H., Barlough, J.E., Madigan, J.E., 1998. Production and characterization of Ehrlichia risticii, the agent of Potomac horse fever, from snails (Pleuroceridae: Juga spp.) in aquarium culture and genetic comparison to equine strains. Journal of Clinical Microbiology 36, 1501–1511.

The authors have no ﬁnancial or personal relationship with other people or organizations that could inappropriately inﬂuence or bias the content of the paper.
Short Communication

# Detection of Neorickettsia risticii from various freshwater snail species collected from a district irrigation canal in Nevada County, California

Nicola Pusterla a,⇑, Daniel Hagerty , Samantha Mapes , Josh VanGeem , Lindsey T. Groves b, Mario Dinucci , Langdon C. Fielding , Jill C. Higgins d

 Department of Medicine and Epidemiology, School of Veterinary Medicine, University of California, Davis, CA 95616, USA  Natural History Museum of Los Angeles County, Los Angeles, CA 90007, USA  Sierra Equine, Grass Valley, CA 95945, USA d Loomis Basin Equine Medical Center, Loomis, CA 95650, USA

## a r t i c l ei n f o

## a b s t r a c t

Article history: Accepted 26 February 2013 Keywords: Neorickettsia risticii Equine neorickettsiosis Horse Freshwater snails Irrigation canal Potomac horse fever This study investigated the role of a district irrigation canal in Nevada County, California, USA, as the point source of infection for Neorickettsia risticii, causative agent of equine neorickettsiosis (EN). A total of 568 freshwater snails comprising spp., (Carpenter, 1857) (Rough Rams- horn), (Gould, 1855) (Protean Physa) and feces from three horses with EN were collected and tested for  $N.$  risticii by real-time PCR. A total of four freshwater snails tested PCR positive for N. risticii. Phylogenetic analysis showed  $99.8–100\%$  homology between the different snail and horse N. risticii iso- lates. This study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and sug- gests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

 



2013 Elsevier Ltd. All rights reserved. $@$  Neorickettsia risticii is the etiologic agent of equine neoricketts- iosis (EN), or Potomac horse fever (PHF), an acute enterocolitis of horses. has recently been associated with trem- atode-vectors, which use freshwater snails and aquatic insects as intermediate hosts and insectivorous birds and bats as deﬁnitive hosts (Pusterla et al., 2003; Gibson et al., 2005). In Northern Cali- fornia, the species of snail incriminated in the life cycle of the hel- minth vector infected with  $N.$  risticii has been identiﬁed as Juga yrekaensis (Henderson, 1935, [Yreka juga]), a common pleurocerid snail, which inhabits fresh or brackish stream water in the North- western United States (Barlough et al., 1998). Additionally, DNA from N. risticii has been detected in virgulate cercariae in lymnaeid snails (Stagnicola spp.) from Northern California, in virgulate xiph- idiocercariae isolated from pleurocerid snails (Elimia livescens, (Menke, 1830), [Liver Elimia]) in central Ohio, and from pleuroce- rid snails (Elimia virginica, (Say, 1817), [Piedmont Elimia]) in cen- tral Pennsylvania, indicating that other species of freshwater snails may also harbor infected trematodes (Reubel et al., 1998; Kanter et al., 2000).



In recent years, there have been an increasing number of re- ported EN cases from Nevada County, California. Of epidemiologi- cal interest is the fact that water is supplied to the premises where conﬁrmed EN cases have historically been diagnosed through a dis- trict irrigation canal system. The aim of this study was, therefore, to determine if freshwater snails collected from these canals har- bored N. risticii.



A total of 568 freshwater snails, including 218 Juga spp., 140 Planorbella subcrenata and 210 Physella virgata (Fig. 1), were collected from ﬁve different sites along the irrigation canal from September 2010 to August 2011 (Fig. 2). Each snail was extracted from its shell using clean forceps and scalpel blades, and placed into a 96-well plate containing  $500\,\upmu\mathrm{L}$ of lysis solution. All extrac- tion steps were performed in a hood with laminar ﬂow. Proteinase K and two grinding beads ( $\mathrm{\Omega}.4\,\mathrm{mm}$  diameter, stainless steel beads, SpexCertiprep) were added and the tissues homogenized in a GenoGrinder2000 (SpexCertiprep) for 2 min at 1000 strokes per minute.

Protein digest was at  $56\,^{\circ}C$ for  $30\,\mathrm{min}$ followed by a  $30\,\mathrm{min}$ per- iod at  $-20\,^{\circ}\mathrm{C}$ to reduce foam. Two hundred microliters of the mix- ture were processed for DNA puriﬁcation using an automated extraction protocol as previously reported (Pusterla et al., 2000a). Extracted nucleic acids were tested for the presence of  $N.$  risticii by real-time PCR using a previously established and validated pro- tocol (Pusterla et al., 2000a). Known positive and negative controls were used with every PCR run. Furthermore, a real-time PCR assay targeting the universal eukaryotic 18S rRNA gene was used as quality control (i.e. efﬁciency of DNA puriﬁcation and ampliﬁca- tion) and as an indicator of inhibition (Eukaryotic 18S rRNA Endog- enous Control, Applied Biosystems).

Three horses diagnosed with PHF from different localities in Ne- vada County were used for phylogenetic analysis. A 527 nucleotide segment of the 16S rRNA gene of N. risticii was generated from the snail and horse isolates in order to determine phylogenetic rela- tionship (Reubel et al., 1998).

Nucleic acid extraction was successful in all 568 freshwater snails based on the detection of the 18S rRNA gene. Four freshwa- ter snails tested PCR positive for N. risticii. These included one Juga species (0.4%) and three P. subcrenata (2.1%) snails, but none of the P. virgata snails tested PCR positive for N. risticii. All three horses presented to Loomis Basin Equine Medical Center with classical signs of EN, including depression, anorexia, fever, colic and soft- formed feces tested PCR positive for N. risticii in feces (Bertin et al., 2011). Sequence comparison of a 527 nucleotide segment of the 16S rRNA gene of N. risticii showed 100% homology amongst the snails and between 99.3% and 100.0% homology with the three horses from Nevada County (Fig. 3).

The biology of N. risticii is associated with an aquatic environ- ment, which explains why EN cases have a seasonal occurrence and are commonly observed near waterways in endemic areas (Barlough et al., 1998). This study reports on the detection of N. ris- ticii in freshwater snails and horses from Nevada County, Califor- nia.Thisnewlyrecognizedendemicareaisgeographically distinct from other endemic areas in California, which are located close to the Oregon border. In the rural area of Nevada County water is provided via a canal system from April to October and used mainly for local agriculture, including pasture irrigation. The water originates from Scotts Flat Reservoir and is carried in an open canal system for over 150 miles (ca. 241 km) to approxi- mately 32,000 irrigable acres (ca. 13,000 ha). To the knowledge of the authors, this is the ﬁrst report associating a district irrigation canal system with the spread of N. risticii.

Demonstrating endemic occurrence of N. risticii was based on the detection of the neorickettsial organism in freshwater snails and resident horses with signs compatible with EN. Previous work has shown that the number of PCR positive snails appears to de- pend on the size of the snails, the month of collection, and geo- graphic origin (Pusterla et al., 2000b). In the present study, we detected a prevalence of infected freshwater snails ranging from  $0.4\%$  to  $2.1\%$  which is in agreement with previous reports (Barlough et al., 1998; Pusterla et al., 2000b). is considered the main species of freshwater snails associated with the life cycle of the digenetic trematode vectors of  $N.$  risticii in Northern California (Reubel et al., 1998). This was in contrast with the present study, where the majority of the PCR positive freshwater snails belonged to P. subcrenata. This planorbid snail is common in the Northwest- ern United States and can be recognized by its sinistrally-coiled shell (Burch, 1989). Previous work has shown that in endemic areas various freshwater snail species can harbor (Reubel et al., 1998).

EN was diagnosed in the horses from Nevada County based on clinical signs, laboratory detection of  $N.$  risticii via PCR and re- sponse to treatment with oxytetracycline and supportive care as previously reported (Bertin et al., 2011). All the affected horses had direct access to irrigated pastures and lived  $100{-}300\,\mathrm{m}$  from the nearest irrigation canal. The close relationship between fresh- water snail and horse  $N.$  risticii isolates was further supported by the high sequence homology of the partial sequence of the 16S rRNA gene of  $N.$  risticii (Reubel et al., 1998). The study represents the ﬁrst report of infection with N. risticii in Planorbella subcrenata and suggests that the irrigation canal was the aquatic environment responsible for the spread of  $N.$  risticii.

### Conﬂict of interest statement

### Acknowledgement

The authors wish to thank Edward J. Johannes (Deixis Consul- tants, SeaTac, WA) for conﬁrming the identiﬁcation of the Juga specimens.

### References

Barlough, J.E., Reubel, G.H., Madigan, J.E., Vredevoe, L.K., Miller, P.E., Rikihisa, Y., 1998. Detection of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Pleuroceridae: Juga spp.) from northern California. Applied and Environmental Microbiology 64, 2888–2893.

Bertin, F.R., Reising, A., Slovis, N.M., Constable, P.D., Taylor, S.D., 2011. Factors associated with survival in 50 horses with Potomac horse fever. In: Proceedings of the Annual Meeting of the American College of Veterinary Internal Medicine, New Orleans, USA, pp. 263.

Burch, J.B., 1989. North American Freshwater Snails. Malocological Publications, Hamburg, MI, USA, pp. 365.

Gibson, K.E., Rikihisa, Y., Zhang, C., Martin, C., 2005. is vertically transmittedinthe trematodeAcanthatriumoregonenseand horizontally transmitted to bats. Environmental Microbiology 7, 203–212.

Kanter, M., Mott, J., Ohashi, N., Fried, B., Reed, S., Lin, Y.C., Rikihisa, Y., 2000. Analysis of 16S rRNA and 51-kilodalton antigen gene and transmission in mice of Ehrlichia risticii in virgulate trematodes from Elimia livescens snails in Ohio. Journal of Clinical Microbiology 38, 3349–3358.

Pusterla, N., Leutenegger, C.M., Sigrist, B., Chae, J.S., Lutz, H., Madigan, J.E., 2000a. Detection and quantitation of Ehrlichia risticii genomic DNA in infected horses and snails by real-time PCR. Veterinary Parasitology 90, 129–135.

Pusterla, N., Johnson, E., Chae, J., Pusterla, J.B., DeRock, E., Madigan, J.E., 2000b. Infection rate of Ehrlichia risticii, the agent of Potomac horse fever, in freshwater stream snails (Juga yrekaensis) from northern California. Veterinary Parasitology 92, 151–156.

Pusterla, N., Johnson, E.M., Chae, J.S., Madigan, J.E., 2003. Digenetic trematodes, Acanthatrium sp. and Lecithodendrium sp., as vectors of Neorickettsia risticii, the agent of Potomac horse fever. Journal of Helminthology 77, 335–339.

Reubel, G.H., Barlough, J.E., Madigan, J.E., 1998. Production and characterization of Ehrlichia risticii, the agent of Potomac horse fever, from snails (Pleuroceridae: Juga spp.) in aquarium culture and genetic comparison to equine strains. Journal of Clinical Microbiology 36, 1501–1511.

The authors have no ﬁnancial or personal relationship with other people or organizations that could inappropriately inﬂuence or bias the content of the paper.
Steric Bulk at Cycloartenol Synthase Position 481 Influences Cyclization and Deprotonation

# 2000 Vol. 2, No. 15 2261-2263

Seiichi P. T. Matsuda,\* Lisa B. Darr, Elizabeth A. Hart, Jennifer B. R. Herrera, Kelly E. McCann, Michelle M. Meyer, Jihai Pang, and Hala G. Schepmann

Department of Chemistry and Department of Biochemistry and Cell Biology, Rice UniVersity, 6100 South Main Street, Houston, Texas 77005

## matsuda@rice.edu

### Received May 4, 2000

## ABSTRACT

Cycloartenol synthase converts oxidosqualene to the pentacyclic sterol precursor cycloartenol. An Arabidopsis thaliana cycloartenol synthase Ile481Val mutant was previously shown to produce lanosterol and parkeol in addition to its native product cycloartenol. Experiments are described here to construct Phe, Leu, Ala, and Gly mutants at position 481 and to determine their cyclization product profiles. The Phe mutant was inactive, and the Leu mutant produced cycloartenol and parkeol. The Ala and Gly mutants formed lanosterol, cycloartenol, parkeol, achilleol A, and camelliol C. Monocycles comprise most of the Gly mutant product, showing that an alternate cyclization route can be made the major pathway by a single nonpolar mutation.

Terpene synthases catalyze cationic cyclization and rear- rangement reactions that convert acarbocyclic precursors to carbocyclic products. Four enzyme families provide hundreds of terpene hydrocarbon structures: the higher plant TPS enzymes (which form monoterpenes, sesquiterpenes, and diterpenes), fungal sesquiterpene synthases, bacterial ses- quiterpene synthases, and a prokaryotic and eukaryotic triterpene synthase family that cyclizes either squalene or oxidosqualene. How terpene synthases mediate these reac- tions has recently begun to be investigated by mutagenesis. Sesquiterpene synthases and squalene-hopene cyclases have been modified to make new compounds. We describe here



 J. Am. Chem. Soc. 1996, 118-1564. (b) Cane, D. E.; Xue, Q.; Van Epp, J. E.; Tsantrizos, Y. S. J. Am. Chem. Soc. 1996, 118  J. Am. Chem. Soc. 1999, 121  J. Am. Chem. Soc. 2000, 122-1866. (7) (a) Merkofer, T.; Pale-Grosdemange, C.; Wendt, K. U.; Rohmer, M.;  Tetrahedron Lett. 1999, 40 Hoshino, T. Chem. Commun. 1998, 23-  Tetrahedron Lett. 1999, 40, 6009 Chem. Commun. 1999, 1999-  Biosci. Biotechnol. Biochem. 1999, 63, 2189- FEMS Microbiol. Lett. 2000, 183- 224. (8) Corey, E. J.; Matsuda, S. P. T.; Bartel, B. Proc. Natl. Acad. Sci. U.S.A. 1993, 90-11632.

10.1021/ol006018w CCC: \$19.00© 2000 American Chemical Society Published on Web 07/06/2000

experiments to determine the role of steric bulk at position 481 of Arabidopsis thaliana cycloartenol synthase (At- CAS1).

Cycloartenol synthase cyclizes oxidosqualene to the pro- tosteryl cation, which is then rearranged and deprotonated to give cycloartenol (Scheme 1). Isoleucine 481 (A. thaliana numbering) is strictly conserved in the known cycloartenol synthases. We recently described an AtCAS1 Ile481Val mutant that converts oxidosqualene to cycloartenol  $(55\%)$ , lanosterol  $(24\%)$ , and parkeol  $(21\%)$ . To investigate further the importance of sterics at this position, we constructed  mutants bearing phenylalanine, leucine, alanine, or $A t C A S1$  glycine at position 481. The AtCAS1 cDNA was subjected to oligo-directed mutagenesis in vector pRS305GAL. The mutants were subcloned into the high-copy yeast expression vector pRS426GAL11 and expressed in the yeast strain LHY2, from which both squalene synthase and lanosterol synthase had been deleted. Typically, cells from 1-L cultures prepared as described were suspended in  $100\,\mathrm{mM}$  sodium phosphate buffer at  $\mathrm{pH}~6.4\$  and lysed in a French Press. Racemic oxidosqualene was added to  $1~\mathrm{mg/mL}$  from a  $20~\mathrm{mg/mL}$  solution in  $20\%$  Triton X-100. The reaction was quenched after  $24\,\mathrm{{h}}$ , and triterpene alcohol products were isolated as described.



Reactions were monitored by TLC (silica gel, 1:1 hexane/ ether), and products were acetylated (acetic anhydride/ pyridine) and identified by GC, GC/MS, and  $^1\mathrm{H}.$ , and/or  $^{13}\mathrm{C}$  NMR. AtCAS1 Ile481Phe was inactive, but the remaining mutants metabolized oxidosqualene. The AtCAS1Ile481Leu products were indistinguishable from cycloartenol by TLC, which cannot resolve most triterpene alcohols because their -hydroxy groups are in sterically similar environments. $3\beta.$  The products were acetylated, and NMR and GC revealed multiple compounds. The  $^{13}\mathrm{C}$  NMR spectrum displayed all 32 cycloartenyl acetate signals, establishing the major product as cycloartenol. Cycloartenyl acetate signals were also dominant in the  $^1\mathrm{H}$  NMR, but signals at  $\delta\,5.22$  (C-11), 0.64 (C-18), and  $0.73~\mathrm{ppm}$  (C-30) and  $^{13}\mathrm{C}$  NMR signals at δ 44.27, 46.99, and 148.11 were characteristic of parkeyl acetate. The  NMR spectrum revealed a trace of $^1\mathrm{H}$  lanosterol, with characteristic lanosteryl acetate signals at  $\delta$  0.68 and  $1.00~\mathrm{ppm}$  corresponding to C18 and C19, respec- tively. GC/MS analyses confirmed the NMR results; the unknowns had retention times (relative to cholesteryl acetate) and fragmentation patterns identical to those of acetylated cycloartenol, parkeol, and lanosterol standards (Scheme 2).



Org. Lett., Vol. 2, No. 15,

The products (percent composition determined by GC) of these enzymes are summarized in Table 1. Even minor steric changes at position 481 impair cycloartenol production. Forming the strained cyclopropane ring is apparently readily compromised. Mutating Ile481 to Leu maintains total steric bulk but shifts a methyl group from the  $\beta\cdot$ - to the  $\gamma.$ -carbon. This isomeric side chain allows deprotonation from C-11 to yield parkeol, but only minimal deprotonation from C-9 or C-8 to form lanosterol. In contrast, mutating to the methyl side chain (Ile481Ala) generates an enzyme that forms lanosterol as its major product. Furthermore, Ala promotes ring B formation less efficiently than Ile, Leu, or Val. The monocyclic cation can deprotonate to either achilleol A or camelliol C. Ile481Gly forms primarily monocycles, and the ratio of achilleol A to camelliol C formation increases relative to that of the Ile481Ala products.

S. cereVisiae lanosterol synthase (ScERG7) mutants were recently described in which Val454 (which corresponds to AtCAS1 Leu481) was mutated to Phe, Leu, Ile, Ala, and Gly. Like AtCAS1, ScERG7 lost activity upon mutating the target residue to Phe. The ScERG7 Val454 Leu and Ile mutants remained accurate lanosterol synthases. The Ala and Gly mutants were lanosterol synthases that also produced achilleol A ( $5\%$  and  $12\%$ , respectively). Decreasing steric bulk at  $S c\mathrm{ERG}7$  position 454 compromised ring B formation, although to a lesser extent than at the analogous AtCAS1 position 481. Moreover, ScERG7Val454 mutants did not form measurable amounts of the alternative deprotonation products cycloartenol, parkeol, and camelliol C  $(<0.5\%)$ . Lanosterol synthase is evidently less sensitive than cy- cloartenol synthase to steric alterations at this position. Perhaps less precise positioning of active site electron density and steric bulk is required to form the energetically favorable tetrasubstituted olefin than to close the strained cyclopropane ring.



These experiments establish that cycloartenol synthase can be mutated to make different tetracyclic or monocyclic compounds in high yield. The mutants described herein may mimic naturally occurring enzymes. Achilleol A and camel- liol C have been isolated from plants, but the enzymes that catalyze their formation remain undiscovered. Several organ- isms20 produce achilleol A without camelliol C, as do the ScERG7 Val454 Ala and Gly mutants. Sasanqua oil21 contains camelliol C and achilleol A, as in the products of AtCAS1 Ile481 Ala and Gly mutants. None of these mutants produced camelliol C without achilleol A, and similarly, camelliol C has never been isolated from a natural source unaccompanied by achilleol A. It is tempting to speculate that camelliol C may be produced in nature by a cycloartenol synthase Ile481 Ala or Gly mutant, and achilleol A from a similar mutant of a different oxidosqualene cyclase.



Acknowledgment. Welch Scholars, and M.M.M was an American Society of Pharmacognosy Undergraduate Fellow. The National Insti- tutes of Health (AI41598) and the Robert A. Welch Founda- tion (C-1323) funded this research. J.B.R.H. was funded by NIH training grant T32 GM08362, and J.P. was supported by NIH grant HL49122.
Steric Bulk at Cycloartenol Synthase Position 481 Influences Cyclization and Deprotonation

# 2000 Vol. 2, No. 15 2261-2263

Seiichi P. T. Matsuda,\* Lisa B. Darr, Elizabeth A. Hart, Jennifer B. R. Herrera, Kelly E. McCann, Michelle M. Meyer, Jihai Pang, and Hala G. Schepmann

Department of Chemistry and Department of Biochemistry and Cell Biology, Rice UniVersity, 6100 South Main Street, Houston, Texas 77005

## matsuda@rice.edu

### Received May 4, 2000

## ABSTRACT

Cycloartenol synthase converts oxidosqualene to the pentacyclic sterol precursor cycloartenol. An Arabidopsis thaliana cycloartenol synthase Ile481Val mutant was previously shown to produce lanosterol and parkeol in addition to its native product cycloartenol. Experiments are described here to construct Phe, Leu, Ala, and Gly mutants at position 481 and to determine their cyclization product profiles. The Phe mutant was inactive, and the Leu mutant produced cycloartenol and parkeol. The Ala and Gly mutants formed lanosterol, cycloartenol, parkeol, achilleol A, and camelliol C. Monocycles comprise most of the Gly mutant product, showing that an alternate cyclization route can be made the major pathway by a single nonpolar mutation.

Terpene synthases catalyze cationic cyclization and rear- rangement reactions that convert acarbocyclic precursors to carbocyclic products. Four enzyme families provide hundreds of terpene hydrocarbon structures: the higher plant TPS enzymes (which form monoterpenes, sesquiterpenes, and diterpenes), fungal sesquiterpene synthases, bacterial ses- quiterpene synthases, and a prokaryotic and eukaryotic triterpene synthase family that cyclizes either squalene or oxidosqualene. How terpene synthases mediate these reac- tions has recently begun to be investigated by mutagenesis. Sesquiterpene synthases and squalene-hopene cyclases have been modified to make new compounds. We describe here



 J. Am. Chem. Soc. 1996, 118-1564. (b) Cane, D. E.; Xue, Q.; Van Epp, J. E.; Tsantrizos, Y. S. J. Am. Chem. Soc. 1996, 118  J. Am. Chem. Soc. 1999, 121  J. Am. Chem. Soc. 2000, 122-1866. (7) (a) Merkofer, T.; Pale-Grosdemange, C.; Wendt, K. U.; Rohmer, M.;  Tetrahedron Lett. 1999, 40 Hoshino, T. Chem. Commun. 1998, 23-  Tetrahedron Lett. 1999, 40, 6009 Chem. Commun. 1999, 1999-  Biosci. Biotechnol. Biochem. 1999, 63, 2189- FEMS Microbiol. Lett. 2000, 183- 224. (8) Corey, E. J.; Matsuda, S. P. T.; Bartel, B. Proc. Natl. Acad. Sci. U.S.A. 1993, 90-11632.

10.1021/ol006018w CCC: \$19.00© 2000 American Chemical Society Published on Web 07/06/2000

experiments to determine the role of steric bulk at position 481 of Arabidopsis thaliana cycloartenol synthase (At- CAS1).

Cycloartenol synthase cyclizes oxidosqualene to the pro- tosteryl cation, which is then rearranged and deprotonated to give cycloartenol (Scheme 1). Isoleucine 481 (A. thaliana numbering) is strictly conserved in the known cycloartenol synthases. We recently described an AtCAS1 Ile481Val mutant that converts oxidosqualene to cycloartenol  $(55\%)$ , lanosterol  $(24\%)$ , and parkeol  $(21\%)$ . To investigate further the importance of sterics at this position, we constructed  mutants bearing phenylalanine, leucine, alanine, or $A t C A S1$  glycine at position 481. The AtCAS1 cDNA was subjected to oligo-directed mutagenesis in vector pRS305GAL. The mutants were subcloned into the high-copy yeast expression vector pRS426GAL11 and expressed in the yeast strain LHY2, from which both squalene synthase and lanosterol synthase had been deleted. Typically, cells from 1-L cultures prepared as described were suspended in  $100\,\mathrm{mM}$  sodium phosphate buffer at  $\mathrm{pH}~6.4\$  and lysed in a French Press. Racemic oxidosqualene was added to  $1~\mathrm{mg/mL}$  from a  $20~\mathrm{mg/mL}$  solution in  $20\%$  Triton X-100. The reaction was quenched after  $24\,\mathrm{{h}}$ , and triterpene alcohol products were isolated as described.



Reactions were monitored by TLC (silica gel, 1:1 hexane/ ether), and products were acetylated (acetic anhydride/ pyridine) and identified by GC, GC/MS, and  $^1\mathrm{H}.$ , and/or  $^{13}\mathrm{C}$  NMR. AtCAS1 Ile481Phe was inactive, but the remaining mutants metabolized oxidosqualene. The AtCAS1Ile481Leu products were indistinguishable from cycloartenol by TLC, which cannot resolve most triterpene alcohols because their -hydroxy groups are in sterically similar environments. $3\beta.$  The products were acetylated, and NMR and GC revealed multiple compounds. The  $^{13}\mathrm{C}$  NMR spectrum displayed all 32 cycloartenyl acetate signals, establishing the major product as cycloartenol. Cycloartenyl acetate signals were also dominant in the  $^1\mathrm{H}$  NMR, but signals at  $\delta\,5.22$  (C-11), 0.64 (C-18), and  $0.73~\mathrm{ppm}$  (C-30) and  $^{13}\mathrm{C}$  NMR signals at δ 44.27, 46.99, and 148.11 were characteristic of parkeyl acetate. The  NMR spectrum revealed a trace of $^1\mathrm{H}$  lanosterol, with characteristic lanosteryl acetate signals at  $\delta$  0.68 and  $1.00~\mathrm{ppm}$  corresponding to C18 and C19, respec- tively. GC/MS analyses confirmed the NMR results; the unknowns had retention times (relative to cholesteryl acetate) and fragmentation patterns identical to those of acetylated cycloartenol, parkeol, and lanosterol standards (Scheme 2).



Org. Lett., Vol. 2, No. 15,

The products (percent composition determined by GC) of these enzymes are summarized in Table 1. Even minor steric changes at position 481 impair cycloartenol production. Forming the strained cyclopropane ring is apparently readily compromised. Mutating Ile481 to Leu maintains total steric bulk but shifts a methyl group from the  $\beta\cdot$ - to the  $\gamma.$ -carbon. This isomeric side chain allows deprotonation from C-11 to yield parkeol, but only minimal deprotonation from C-9 or C-8 to form lanosterol. In contrast, mutating to the methyl side chain (Ile481Ala) generates an enzyme that forms lanosterol as its major product. Furthermore, Ala promotes ring B formation less efficiently than Ile, Leu, or Val. The monocyclic cation can deprotonate to either achilleol A or camelliol C. Ile481Gly forms primarily monocycles, and the ratio of achilleol A to camelliol C formation increases relative to that of the Ile481Ala products.

S. cereVisiae lanosterol synthase (ScERG7) mutants were recently described in which Val454 (which corresponds to AtCAS1 Leu481) was mutated to Phe, Leu, Ile, Ala, and Gly. Like AtCAS1, ScERG7 lost activity upon mutating the target residue to Phe. The ScERG7 Val454 Leu and Ile mutants remained accurate lanosterol synthases. The Ala and Gly mutants were lanosterol synthases that also produced achilleol A ( $5\%$  and  $12\%$ , respectively). Decreasing steric bulk at  $S c\mathrm{ERG}7$  position 454 compromised ring B formation, although to a lesser extent than at the analogous AtCAS1 position 481. Moreover, ScERG7Val454 mutants did not form measurable amounts of the alternative deprotonation products cycloartenol, parkeol, and camelliol C  $(<0.5\%)$ . Lanosterol synthase is evidently less sensitive than cy- cloartenol synthase to steric alterations at this position. Perhaps less precise positioning of active site electron density and steric bulk is required to form the energetically favorable tetrasubstituted olefin than to close the strained cyclopropane ring.



These experiments establish that cycloartenol synthase can be mutated to make different tetracyclic or monocyclic compounds in high yield. The mutants described herein may mimic naturally occurring enzymes. Achilleol A and camel- liol C have been isolated from plants, but the enzymes that catalyze their formation remain undiscovered. Several organ- isms20 produce achilleol A without camelliol C, as do the ScERG7 Val454 Ala and Gly mutants. Sasanqua oil21 contains camelliol C and achilleol A, as in the products of AtCAS1 Ile481 Ala and Gly mutants. None of these mutants produced camelliol C without achilleol A, and similarly, camelliol C has never been isolated from a natural source unaccompanied by achilleol A. It is tempting to speculate that camelliol C may be produced in nature by a cycloartenol synthase Ile481 Ala or Gly mutant, and achilleol A from a similar mutant of a different oxidosqualene cyclase.



Acknowledgment. Welch Scholars, and M.M.M was an American Society of Pharmacognosy Undergraduate Fellow. The National Insti- tutes of Health (AI41598) and the Robert A. Welch Founda- tion (C-1323) funded this research. J.B.R.H. was funded by NIH training grant T32 GM08362, and J.P. was supported by NIH grant HL49122.
# open

### Received: 13 June 2019

### Accepted: 28 August 2019

### Published: xx xx xxxx

# Stable fabrication of a large  nanopore by controlled dielectric  breakdown in a high-pH solution  for the detection of various-sized  molecules

itaru Yanagi, Rena Akahori & Ken-ichi takeda for nanopore sensing of various-sized molecules with high sensitivity, the size of the nanopore should  be adjusted according to the size of each target molecule. for solid-state nanopores, a simple and  inexpensive nanopore fabrication method utilizing dielectric breakdown of a membrane is widely used.  This method is suitable for fabricating a small nanopore. However, it suffers two serious problems when  attempting to fabricate a large nanopore: the generation of multiple nanopores and the non-opening  failure of a nanopore. in this study, we found that nanopore fabrication by dielectric breakdown of a  Sin membrane under high-pH conditions  $(\mathsf{p H}\geq11.3)$  could overcome these two problems and enabled  the formation of a single large nanopore up to  $40\,{\mathsf{n m}}$  in diameter within one minute. Moreover, the  ionic-current blockades derived from streptavidin-labelled and non-labelled DnA passing through the  fabricated nanopore were clearly distinguished. the current blockades caused by streptavidin-labelled  DNA could be identified even when its concentration is  $\mathbf{1}\%$ of the total DNA.



Nanopores have been widely used in recent years as a highly sensitive microscope that can observe various bio- molecules in an aqueous solution. The structural and electrical characteristics of the target molecules can be  extracted by detecting the changes in ionic currents through the nanopore when the molecules pass through it.  For instance, single-molecule DNA sequencing is currently possible–14 by using a biological nanopore with a  diameter of 2 nm or less. In addition, the detection of proteins, antigen-antibody complexes and probe-labelled  DNA is possible–21 by using a solid-state nanopore with a diameter of several to tens of nanometres. Moreover,  the detection of various viruses is also possible–27 by using a solid-state nanopore with a diameter of several tens  to hundreds nanometers. To detect such varied molecules with high sensitivity, adjusting the size of the nanopore  according to the size of each target molecule is important.

For solid-state nanopores, a nanopore with a diameter of a few to hundreds of nanometres can be fabricated  by choosing an appropriate manufacturing process–38. This capability is an advantage over biological nanopores  whose diameters are typically limited to approximately 2 nm or less. A nanopore with a diameter of approximately   $20\,\mathrm{nm}$  to several hundred nanometres can be fabricated by utilizing optical or electron-beam (EB) lithography  followed by reactive-ion etching. An even smaller nanopore down to 1 nm in diameter can be fabricated by  drilling a membrane with a focused-electron or helium-ion beam30–34. In addition, dielectric breakdown of a  membrane has been widely utilized as a method of nanopore fabrication in recent years35–61 because of its sim- plicity and inexpensiveness. With the application of a high constant voltage to a membrane and termination of the  voltage when the current between the electrodes reaches a predetermined cut-off value, a nanopore can be created  in a membrane. This method is called controlled breakdown (CBD)35,,, which also enables the fabrication of a  small nanopore down to 1 nm in diameter.

CBD, however, suffers two problems for the fabrication of a single large nanopore with a diameter of larger  than approximately 5 nm. One is the generation of multiple nanopores, which has been reported in several  studies,,,. Our observed example of the generation of multiple nanopores in a 10-nm-thick SiN membrane is  also shown in Supplementary Fig. 1. This behaviour is caused by the second and the subsequent dielectric break- downs of the membrane occurring until the current reaches the cut-off current after the first dielectric breakdown  (i.e., multiple nanopores generate while widening a nanopore generated at the first dielectric breakdown). When  trying to fabricate a large nanopore, it is necessary to set a high cut-off current value. As a result, the time spent on  the nanopore widening process increases, leading to a high risk of the generation of multiple nanopores.



This problem seems to be solved if the time of the nanopore widening process, compared to the time to the  first dielectric breakdown, can be sufficiently shortened. In this context, a thicker membrane is expected to be  suitable for the fabrication of a single large nanopore. A higher voltage is required to break down a thicker mem- brane. Consequently, a higher Joule energy is released after the first dielectric breakdown, which contributes to  faster nanopore enlargement. However, the second problem appears here. As we reported previously, when  CBD was applied to a thick (20-nm-thick) SiN membrane, a nanopore could not be fabricated, and only a local  conductive-film portion was created. With an increase in the cut-off current value, the area of the conductive-film  portion expanded, and the membrane was eventually destroyed. Therefore, the fabrication of a single large nan- opore by CBD has been very difficult.

Recently, it has been reported that the laser-assisted CBD can form a large nanopore with a diameter of  . While this technique is revolutionary, it requires a laser optical system for nanopore fabrication. In  $20–50\,\mathrm{{nm}^{57}}$  this study, we report a method that enables the fabrication of a single large nanopore in a thick membrane by CBD  without the need for optics. The application of CBD to a SiN membrane with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  in an  aqueous solution with a pH higher than 11.3 enabled the fabrication of a nanopore instead of a conductive-film  portion. The diameter of the fabricated nanopore could be roughly controlled within a range from  $5\,\mathrm{nm}$  to  $40\,\mathrm{nm}$   by changing the cut-off current value. In addition, the occurrences of streptavidin (SA)-labelled and non-labelled  DNA passing through a nanopore were clearly discriminated by monitoring the difference in ionic-current block- ades derived from those molecules. Even if the SA-labelled DNA was present in only  $1\%$  of the total DNA in an  aqueous solution, its presence could be detected.

Results The schematic setup for the dielectric breakdown experiments is presented in Fig. 1(a). The area of the SiN mem- brane with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  was restricted within approximately a small square area of approx- imately  $600\times600~\mathrm{nm}^{2}$  so that fabricated nanopores could be easily found. Two  $\mathrm{Ag/AgCl}$ electrodes (cis and  trans electrodes) were immersed in 1 M KCl aqueous solution at various  $\mathrm{pH}$  values for applying voltages and  measuring currents through the membrane. In this study, SiN membranes with thicknesses of  $20\,\mathrm{nm}$  were mainly  used for the dielectric breakdown experiments. Figure 1(b) presents examples of current-time traces when CBD  was applied to 20-nm-thick SiN membranes under ten different pH conditions. The voltage applied was 20 V  ( $V_{\mathrm{trans}}\,{=}\,0\,\mathrm{V}$  and  $V_{\mathrm{cis}}\,{=}\,20\,\mathrm{V})$ , and the cut-off current  $(I_{\mathrm{cutoff}})$  was set at  $1\,\upmu\mathrm{A}.$   $I_{\mathrm{cutoff}}$ was set as a limit of the output  current of the measuring instrument, and the voltage automatically dropped when the current between the elec- trodes reached $I_{\mathrm{cutoff}}\pm0.1\%$ . The dielectric breakdown point was clearly confirmed in each time trace. Figure 1(c)  presents the plots of the time to breakdown (TBD) under each pH condition. The TBD became shorter as the  aqueous solution became more acidic or alkaline. In particular, faster breakdowns were prominently observed  under alkaline conditions than under neutral or acidic conditions. Therefore, Si-N bond breakage in the mem- brane was thought to be strongly promoted by electrochemical reactions induced particularly by hydroxide ions.  Notably, this trend is opposite to that reported by Kwok et al.. According to their report, the TBD of SiN mem- branes at a  $\mathrm{pH}$  of 2 was approximately 100 times shorter than that at a pH of 13.5. In addition, the longest TBD  was observed at a pH of approximately 10 in their report, while it was observed at a pH of approximately 5 in our  result. These differences are thought to be due to the difference in deposition conditions of the SiN membrane  layers, which affects their basic material properties. Details of the formation process of our SiN membranes are  described in the Methods section. Incidentally, the reason for shorter TBD under acidic condition is thought to  be due to $\mathrm{H^{+}}$  or hole injection into the membrane which strongly promoted impact ionization-induced avalanche  and Si-N bond breaking,.

Figure 2 presents transmission electron microscopy (TEM) images of ten different 20-nm-thick SiN mem- branes after CBD under ten different pH conditions. During CBD, the applied voltage and  $I_{\mathrm{cutoff}}$ were set at  $20\,\mathrm{V}$  ( $V_{\mathrm{trans}}\,{=}\,0\,\mathrm{V}$  and  $V_{\mathrm{cis}}\,{=}\,20\,\mathrm{V})$  and  $1\,\upmu\mathrm{A},$  respectively. An image of the entire membrane and a magnified view of the  area around the created defect are shown in each image set. From the entire images of the membranes, the num- ber of created defects (indicated by yellow arrows) was confirmed to be one per membrane. The magnified views  of the defects confirmed that there were two kinds of cases: instead of a nanopore, a local conductive-film portion  was created (i.e., amorphous material was confirmed in the defect area), or a nanopore was created (i.e., no amor- phous material was confirmed in the defect area). The conductive-film portion is thought to be a low-density film  that allows ion conduction. We previously reported that only a conductive-film portion, not a nanopore, was cre- ated when CBD was applied to a 20-nm-thick SiN membrane at a pH of 7.556. In the present experiment, the same  result was obtained even when CBD was performed in a pH range from 1 to 11. On the other hand, interestingly, a  nanopore was created when CBD was performed in the solution with a pH higher than 11.3. In addition, the size  of the nanopore created increased as the  $\mathrm{pH}$ of the solution increased. We subjected at least three membranes to  CBD at each pH and confirmed the same trend described above. This result suggests that the membrane could be  etched by hydroxide ions in the process of dielectric breakdown. Generally, it is known that SiN is hardly etched  and Si is easily etched in an alkaline aqueous solution,. Therefore, the membrane was assumed to be etched by  the following two steps: (i) Si-N bonds were destabilized by a high electric field during CBD, and the number of Si  dangling bonds and Si-Si bonds increased. (ii) Si was etched by the following chemical reaction,:

## 
$$
\mathrm{Si\mathrm{~+~}4O H^{-}\rightarrow S i(O H)_{4}\mathrm{~+~}4e^{-}},
$$


## 
$$
2\mathrm{H}_{2}\mathrm{O}+2\mathrm{e}^{-}\rightarrow2\mathrm{OH}+\mathrm{H}_{2}.
$$


(1) (2)



Figure 3(a) presents the corresponding I-V characteristics for each membrane after CBD. Figure 3(b) presents  the pH dependence of  $I_{\mathrm{rec}}\,{=}\,\big|I(0.4\,\mathrm{V})/I(-0.4\,\mathrm{V})\big|,$  which represents the degree and direction of each  $I^{-}V$ curve’s  rectification. In the case of  $\mathrm{pH}\!\geq\!11.3$  where a nanopore was created, each  $I^{-}V$ curve showed almost ohmic char- acteristics, and  $I_{\mathrm{rec}}$  was approximately 1. On the other hand, in the case of  $\mathrm{pH}\,1$  to 11 where a nanopore was not  created, the shape of the  $I^{-}V$ curve and  $I_{\mathrm{rec}}$  varied in accordance with the  $\mathrm{pH}$  value. A convex upward I-V curve  (i.e.,  $I_{\mathrm{rec}}\!<\!1)$  at  $\mathrm{pH}\;1\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;$ , an almost ohmic  $I^{-}V$ curve (i.e.,  $I_{\mathrm{rec}}\!=\!1)$  at  $\mathrm{pH}\,3$ , and convex downward I-V curves (i.e.,   at  $\mathrm{pH}\,5\,\$  to 11 were confirmed. $I_{\mathrm{rec}}\!>\!1)$  $I_{\mathrm{rec}}$  monotonically increased with an increase in  $\mathrm{pH}$ .

This result provided us the distribution of conduction carriers (i.e., conduction ions) in the conductive-film  portion at each  $\mathrm{pH}$  without voltage biases. Generally, a rectification of an  $I^{-}V$ curve appears when there is an  asymmetric carrier distribution in the direction of current flow. Semiconductor PN junction devices are the  most well-known examples. Recently, it has been pointed out that such a rectification also occurs in a current  through a conical nanopore, or nanofluidic channel, due to the asymmetric carrier distribution. Figure 3(c)  presents the assumed conduction-ion distributions in the conductive-film portions. The density of negative con- duction ions was thought to be high on the trans side of the conductive-film portion at  $\mathrm{pH}\,1$  because  $I_{\mathrm{rec}}\!<\!1$  and   was a forward-biased state. In the case of pH 3,  $I_{\mathrm{rec}}$  $V_{\mathrm{trans}}<V_{\mathrm{cis}}$ was approximately 1, which suggested that there  was almost no asymmetry in the distribution of conduction ions. On the other hand, the density of positive  conduction ions was thought to be high on the trans side of the conductive-film portion at  $\mathrm{pH}\,5$  to 11 because   and  $I_{\mathrm{rec}}\!>\!1$  $V_{\mathrm{trans}}<V_{\mathrm{cis}}$  was a reverse-biased state. These positively and negatively biased carrier distributions were  thought to be induced because the conductive-film portion itself was negatively or positively charged depending  on the pH value; that is, the isoelectric point of the conductive-film portion was assumed to be approximately 3,  and the portion was positively or negatively charged when the pH of the solution was smaller or larger than pH 

## 
$$
d_{\mathrm{TEM}}=2\bigg(\frac{S_{\mathrm{TEM}}}{\pi}\bigg)^{1/2},
$$


(3) where  $S_{\mathrm{TEM}}$  is the area of the nanopore surrounded by the yellow line in each TEM image, as shown in Fig. 5(a).  The area was measured using image processing software (ImageJ). Figure 5(b) presents the relationship between   and $d_{\mathrm{{TEM}}}$  $I_{\mathrm{cutoff}}\left(N\!=\!24\right)$ . Nanopores with a  $d_{\mathrm{TEM}}$  of approximately 5 to  $40\,\mathrm{nm}$  could be fabricated by changing the  value of  $I_{\mathrm{cutoff}}$ from 0.1 to  $6.5\,\upmu\mathrm{A}$ . Note that the number of nanopores observed was always one per membrane.  Figure 5(c) presents the relationship between the conductance  $(G)$  of the ionic current through the nanopore  and  $d_{\mathrm{TEM}}$ . The solid line represents the theoretically calculated relation assuming a cylindrical nanopore, which  is obtained as follows–70:



## 
$$
G=\sigma\bigg(\frac{4h_{\mathrm{eff}}}{\pi d^{2}}+\frac{1}{d}\bigg)^{-1},
$$


(4) where  $\upsigma\,{=}\,0.123\,\mathrm{S/cm}$  is the measured conductance of a KCl aqueous solution with a pH of 12.7 at  $24\,^{\circ}\mathrm{C},$   $d$ is the  diameter of the nanopore, and  $h_{\mathrm{eff}}$ is the height of the nanopore (often called the effective thickness of the nanop- ore). The plotted measurements could be well fitted with the theoretically calculated line by setting  $h_{\mathrm{eff}}$ at  $12.5\,\mathrm{nm}$ .  According to many previous reports,,–71,  $h_{\mathrm{eff}}$ is smaller than the actual membrane thickness. The result in this  study is also consistent with this trend. The times required to form the nanopores by CBD were less than 1 min  regardless of the size of the nanopore (see Supplementary Fig.  $4(\mathrm{a}).$ ).



CBD under high-pH conditions was also effective for nanopore fabrication in 14-nm-thick SiN membranes  (Fig. 6). The current-time traces during CBD and images of the membrane after CBD for each case at  $\mathrm{pH}\,7.5$ and  12.7 are presented in Fig. 6(a,b). Similar to the case of the  $20\mathrm{-nm}$ -thick SiN membranes, CBD in a 14-nm-thick  SiN membrane provided a conductive-thin portion instead of a nanopore at  $\mathrm{pH}\,7.5$ , whereas a nanopore was cre- ated in the case of  $\mathrm{pH}\,12.7\,\$ . Figure  $6(c)$  presents the dependence of  $d_{\mathrm{TEM}}$ on  $I_{\mathrm{cutoff}}\left(N\!=\!6\right)$ . All the nanopores were  fabricated by CBD at an applied voltage of 11 V and  $\mathrm{pH}$  of 12.7, and the  $d_{\mathrm{{TEM}}}$ became larger with an increase in  . Figure  $I_{\mathrm{cutoff}}.$ 6(d) presents the relationship between  $G$  and  $d_{\mathrm{{TEM}}}$ . The effective thickness  $(h_{\mathrm{eff}})$  was estimated to be  To identify the current-blockade signals derived from SA-labelled dsDNA, free unbound SA should not pass  through the nanopore. Therefore, we first examined whether free SA can pass through the nanopore. Figure  $^{7(\mathrm{b},\mathrm{c})}$   present respectively the time traces of ionic currents through the nanopores fabricated in the 20-nm- and  14-nm-thick SiN membranes when the aqueous solution in the cis chamber contained  $100\,\mathrm{nM}$ SA. Almost no  ionic-current blockades derived from SA translocation through the nanopores were confirmed (approximately  less than 5 events/min). In the following experiments, the concentration of free SA in aqueous solution did not  exceed  $100\,\mathrm{nM}$ . Figure 7(d) presents the ionic current through the nanopore fabricated in the 20-nm-thick  SiN membrane when the aqueous solution in the cis chamber contained 15 nM 400-bp dsDNA. Numerous  ionic-current blockades caused by dsDNA translocations through the nanopore were observed. After the meas- urement, the solution in the cis chamber was partly displaced by the solution of 1 M KCl with  $15\,\mathrm{nM}$ SA-labelled  80-bp dsDNA so that SA-labelled and non-labelled dsDNA could be mixed in the cis chamber. Figure 7(e) pre- sents the ionic current through the nanopore after SA-labelled and non-labelled DNA were mixed in the cis  chamber. Larger current blockades than those derived from dsDNA were frequently observed, which indicated  the passage of SA-labelled dsDNA through the nanopore.



Magnified views of the typical current blockades when SA-labelled and non-labelled dsDNA passed through  the nanopore are presented in Fig. 7(f). The dwell times of the current blockades derived from SA-labelled 80-bp  dsDNA were typically shorter than those derived from 400-bp dsDNA, reflecting the difference in the dsDNA  lengths. The scatter plots of the dwell time and  $\Delta I_{\mathrm{max}}$ and the corresponding histograms of before and  $\Delta I_{\mathrm{max}}$  after the mixture of SA-labelled and non-labelled dsDNA are presented in Fig.  $7(\mathrm{g-j})$ .  $\Delta I_{\mathrm{max}}$  was determined as  the maximum current-blockade value observed in each current-blockade event. The figures confirmed that the  occurrences of SA-labelled and non-labelled dsDNA passing through the nanopore could be clearly distinguished  based on the value of  $\Delta I_{\mathrm{max}}$ .

Next, we examined the event discrimination of SA-labelled and non-labelled 400-bp dsDNA passing through  nanopores fabricated in 14-nm-thick SiN membranes (Fig. 8). Nine different aqueous solutions for the cis cham- ber were prepared; that is, a solution of 1 M KCl with  $15\,\mathrm{nM}$  SA-labelled 400-bp dsDNA and a solution of 1 M KCl  with  $15\,\mathrm{nM}\,400.$ -bp dsDNA were mixed at ratios of 1:0, 9:1, 7:3. 5:5, 3:7, 1:9, 5:95, 1:99 and 0:1. Figure 8(a) pre- sents five results from the nine experiments. Compared to the results shown in Fig. 7(h,j), these scatter plots and   histograms exhibit less clear boundaries between the two events derived from SA-labelled and non-labelled  $\Delta I_{\mathrm{max}}$  dsDNA. We cannot currently provide a clear explanation for this difference. Possible causes include difference in  the thicknesses of the SiN membranes ( $20\,\mathrm{nm}$ or  $14\,\mathrm{nm}$ ) and lengths of dsDNA (80 bp or 400 bp). We provision- ally determined the boundary value of  $\Delta I_{\mathrm{max}}$ derived from SA-labelled and non-labelled dsDNA as  $c+4w,$  where   is the central value of a Gaussian curve fitted to each spectrum around  $c$  $\Delta I_{\mathrm{max}}\,{=}\,0.5\,\mathrm{nA}$ , which was attributed to  the events mostly caused by dsDNA translocation through the nanopore, and  $w$ is the value of the full width at  half maximum (FWHM) of the Gaussian curve.

Small current blockades derived from dsDNA translocation through the nanopore were observed even in the  case of SA-labelled dsDNA:  $\mathrm{{dsDNA}}\,{=}\,1{:}\,0$ due to the remaining dsDNA unreacted with SA. On the other hand,  large current blockades derived from SA-labelled dsDNA translocation through the nanopore could be clearly  observed even in the case of SA-labelled dsDNA:  $\mathrm{dsDNA}\,{=}\,1{:}99\$ . Figure 8(b) presents magnified views of the typ- ical current blockades derived from SA-labelled 400-bp dsDNA. Each waveform consists of two current-blockade  levels, which were not observed when SA-labelled 80-bp dsDNA passed through the nanopore (Fig. 7(f)). Each  of the two levels is believed to be attributable to the partial entry of dsDNA or SA into the nanopore. Figure 8(c)  presents the frequency ratio of the current-blockade events derived from SA-labelled and non-labelled dsDNA in  each of the nine experiments. The figure confirms that the event frequency ratio changed, reflecting the mixing  ratio of the two solutions containing SA-labelled dsDNA and dsDNA.

Discussion Stable fabrication of a single large nanopore in a SiN membrane by CBD was examined. First, we applied CBD to  20-nm-thick SiN membranes under various pH conditions. Instead of a nanopore, a local conductive-film por- tion was created in the membrane in the case of  $\mathrm{pH}<11.3$ . On the other hand, a single nanopore was fabricated  when  $\mathrm{pH}\geq11.3$ . This result suggests that the membrane could be etched to form a nanopore by hydroxide ions  under a high electric field. The conduction-ion distribution in the conductive-film portion, incidentally, could  be extrapolated from the shape of the I-V curve, which changed depending on the pH of the aqueous solution.

The size of the nanopore could be controlled by changing the applied voltage or  $I_{\mathrm{cutoff}}$ during CBD at a  $\mathrm{pH}$  of  12.7. For example, nanopores with diameters of 5 to  $40\,\mathrm{nm}$  could be fabricated by changing the value of  $I_{\mathrm{cutoff}}$ from   to  $6.5\,\upmu\mathrm{A}$  $0.1\,\upmu\mathrm{A}$  at an applied voltage of 18 V. The effective thickness of the fabricated nanopores was found to be  approximately  $12.5\,\mathrm{nm}$ , which was estimated from the relationship between the conductance  $(G)$  and equivalent  diameter  $(d_{\mathrm{{TEM}}})$  of the nanopores. In addition, this high-pH CBD method was also effective in nanopore fabri- cation in 14-nm-thick SiN membranes. Nanopores with diameters of 7 to  $12\,\mathrm{nm}$  could be fabricated by changing  from  $0.3\,\upmu\mathrm{A}$  $I_{\mathrm{cutoff}}$  to  $1\,\upmu\mathrm{A}$  at an applied voltage of 11 V, and the effective thickness of their nanopores was found  to be approximately  $11\,\mathrm{nm}$ .

The occurrences of SA-labelled and non-labelled dsDNA passing through the fabricated nanopores could  be distinguished by the difference in ionic-current blockades caused by those two molecules. The ionic-current  blockades derived from SA-labelled dsDNA translocation through the nanopore could be clearly detected even  when the concentration of SA-labelled dsDNA was only  $1\%$  of the total concentration of dsDNA. In addition, the  frequency ratio of the current-blockade events caused by SA-labelled and non-labelled dsDNA changed, reflect- ing the mixing ratio of SA-labelled dsDNA and dsDNA.

The method of CBD under high-pH conditions examined in this study is a simple and rapid (less than 1 min)  nanopore fabrication method that enables the stable fabrication of various-sized solid-state nanopores without  the need for expensive equipment such as EB lithography, TEM or helium-ion microscopy (HIM). In addition,  the fabricated nanopores showed good performance in the detection of different-sized molecules. Consequently,  we believe that this nanopore fabrication method will accelerate the research and development of nanopore  devices for detecting various-sized molecules.



Methods fabrication of membranes. The SiN membranes were fabricated on an 8-inch silicon wafer with a thick- ness of  $725\,\upmu\mathrm{m}$ . The fabrication process was the same as that described in our previous report. First, a SiN  layer with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  was deposited on both sides of the Si wafer via low-pressure chemical  vapour deposition (reacting gases:  $\mathrm{SiH}_{2}\mathrm{Cl}_{2}$ -NH3; flow ratio:  $\mathrm{SiH}_{2}\mathrm{Cl}_{2}{:}\mathrm{NH}_{3}{=}1{:}25$ ;  $770\,^{\circ}\mathrm{C};$  0.5 Torr). The compo- sition ratio of the SiN layer analysed by X-ray photoelectron spectroscopy ( $\mathrm{PHI}\,5000$ VersaProbe II, X-ray: Al   ULVAC-PHI, Inc.) was approximately  $\mathrm{Si}{\cdot}\mathrm{N}\,{=}\,1{:}1{.}24\,\$  $\operatorname{K}_{\mathbb{\alpha},\mathbb{\beta}}$ . Then, a  $\mathrm{SiO}_{2}$  sacrificial layer with a thickness of approx- imately  $260\,\mathrm{nm}$  was deposited on the SiN layer on the front side of the wafer, and a SiN layer with a thickness of  approximately  $90\,\mathrm{nm}$  was deposited on both sides of the wafer. Next, the top SiN layer in each square area approx- imately  $600\times600\,\mathrm{nm}^{2}$  in size and the backside SiN layer in each corresponding  $1038\times1038\,\upmu\mathrm{m}^{2}$  square area were  subsequently etched by reactive-ion etching, followed by etching of the Si substrate with tetramethylammonium  hydroxide (TMAH) at  $85\,^{\circ}\mathrm{C}$ for approximately 9 hours. After etching of the Si substrate, the wafer was diced into  chips. Finally, the  $\mathrm{SiO}_{2}$  sacrificial layer was etched with potassium hydroxide ( $33\,\mathrm{wt.}\%$  solution of KOH at approx- imately  $70\,^{\circ}\mathrm{C}_{\lambda}$ ) before the dielectric breakdown experiment of the SiN membrane.

observation of Sin membranes. Observations of the SiN membranes were performed using a  field-emission TEM (JEM-2100F (HRP),  $200\,\mathrm{kV},$  JEOL, Ltd.). Before the observations, the membranes were  immersed in deionized water for a day to remove any salt residues. The areas of the fabricated nanopores were  measured using image processing software (ImageJ, National Institutes of Health, Bethesda, MD, USA). The  observation process described above was the same as that described in our previous report.

Setup for dielectric breakdown experiments. Initially, the SiN membrane was mounted onto a  custom-built acrylic flow cell. Two chambers (each with a volume of  $90\,\upmu\mathrm{L}$ ) separated by the membrane were  formed in the flow cell: a cis chamber and a trans chamber. Aqueous solutions at various pH values were prepared  by adding KOH or HCl to 1 M potassium chloride,  $10\,\mathrm{mM}$  Tris-HCl, and 1 mM EDTA. The pH of the aqueous  solution was measured using a LAQUA act pH/OPR/COND METER D-74 (HORIBA Scientific, Japan). After  filling the aqueous solution into both chambers, two  $\mathrm{Ag/AgCl}$ electrodes were immersed in both solutions to  ensure electrical contact. Application of a constant voltage, measurement of the current through the membrane  during CBD, and measurement of the I-V characteristics were performed using a 4156B precision semiconductor  parameter analyser (Agilent Technologies, Inc.). The cut-off current for CBD was set as the current compliance  value of the 4156B system.

Preparation of streptavidin-labelled 80-bp dsDNA for measurements. First, an 80-mer single-  stranded DNA (ssDNA) whose  $5^{\prime}$  end was modified with biotin was annealed with a primer. Both the biotin-modified   ssDNA and primer were purchased from Sigma Aldrich, Japan. Then, the primer was extended by an elongation   reaction, and biotin-modified 80-bp dsDNA was created. The sequences of the ssDNA and the primerwere   -ACGAATTCGAGCTCGGTACCCGGGGATCCTCTAGAGTCGACCTGCAGGCATGCAAGCTTGGCA  $5^{\prime}$  CTGGCCGTCGTTTTACA- $3^{\prime}$  and  $5^{\prime}.$ -TGTAAAACGACGGCCAGT- $3^{\prime}$ , respectively. The elongation reaction was   performed in  $50–\upmu\mathrm{L}$  reaction volumes consisting of  $3\,\upmu\mathrm{M}$  primer,  $3\,\upmu\mathrm{M}$  ssDNA,  $320\,\mathrm{U}$  Bst 3.0 DNA polymerase, 1X   isothermal amplification buffer, 1 mM dNTPs and  $1\,\mathrm{{mM}\,\mathrm{{MgSO}_{4}}}$ (New England Biolabs, Ipswich, MA). The time and   temperature of the reaction were 60 min and $65^{\circ}\mathrm{C},$  respectively. The product after the elongation reaction was purified   using NucleoSpin® Gel and PCR Clean-up (Takara Bio Inc, Japan). The purified product was then quantitated with a   Qubit fluorometric system (Life Technologies). A biotin-streptavidin binding reaction was performed in  $18.3\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $1.5\,\upmu\mathrm{M}$  biotin-modified dsDNA and  $10\,\upmu\mathrm{M}$  streptavidin (Thermo Fisher Scientific,  Waltham, MA). The time and temperature of the binding reaction were  $37\,^{\circ}\mathrm{C}$  and  $30\,\mathrm{min}$ , respectively.

Preparation of streptavidin-labelled 400-bp dsDNA for measurements. A portion of λDNA  (NIPPON GENE CO., LTD, Japan) was duplicated by PCR amplification using a biotin-modified forward  primer and a biotin-unmodified reverse primer to obtain biotin-modified 400-bp dsDNA. PCR amplification  was performed using MiniAmp Plus Thermal Cyclers (Thermo Fisher Scientific, Waltham, MA). Both ssDNA  and primer were purchased from Sigma Aldrich, Japan. The sequences of the forward and reverse primers  were  $5^{\prime}$ -TGCAACGAACAGGTCACTATCA- $3^{\prime}$  and  $5^{\prime}$ -GAGCAAAGCAAAACAGGCGTA- $3^{\prime}$ , respectively.  PCR amplification was performed in  $50\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $0.3~{\upmu\mathrm{M}}$  forward primer,  $0.3~{\upmu\mathrm{M}}$  reverse primer,  $50\,\mathrm{ng}$ λDNA, 1 U KOD-Plus- Neo, 1X PCR buffer for KOD -Plus- Neo,  $0.2\,\mathrm{mM}$  dNTPs, and   $1.5\,\mathrm{mM}$   $\mathrm{MgSO}_{4}$ (TOYOBO CO., LTD., Japan). The cycling parameters of PCR were  $94\,^{\circ}\mathrm{C}$ for  $2\,\mathrm{min}\longrightarrow\left[98\,^{\circ}\mathrm{C}\right.$  for  $10\,\mathrm{sec}\!\rightarrow\!58\,^{\circ}\mathrm{C}$  for  $30\,\mathrm{sec}\,{\longrightarrow}\,68\,^{\circ}\mathrm{C}$  for  $30\,\mathrm{sec}]\times35$  times. The product obtained after PCR was purified using  NucleoSpin® Gel and PCR Clean-up (Takara Bio Inc., Japan). The purified product was then quantitated with a  Qubit fluorometric system (Life Technologies). A biotin-streptavidin binding reaction was performed in  $18.3\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $1.5\,\upmu\mathrm{M}$  biotin-modified dsDNA and  $10\,\upmu\mathrm{M}$  streptavidin. The time and tempera- ture of the binding reaction were  $37\,^{\circ}\mathrm{C}$  and  $30\,\mathrm{min}$ , respectively.

Setup for measuring DnA translocation through a nanopore. Prior to the measurements of DNA  translocation through a nanopore, the aqueous solutions in both chambers were displaced by a 1 M KCl aqueous  solution at a  $\mathrm{pH}$  of 8.7. After that, the solution in the cis chamber was displaced by a  $1\,\mathrm{M}\;\mathrm{KCl}$ aqueous solution  containing samples (i.e., SA-labelled dsDNA and dsDNA) at a  $\mathrm{pH}$  of 8.7. The ionic-current measurements were  performed using a patch-clamp amplifier (Axopatch 200B, Axon Instruments, Union City, CA). The detected  current was low-pass-filtered with a cut-off frequency of  $10\,\mathrm{kHz}$  using a four-pole Bessel filter and then digi- tized with an NI USB-6281 18-bit DAQ AD converter (National Instruments, Austin, TX) at  $50\,\mathrm{kHz}$ . Finally,  the current was recorded on the hard disk of a personal computer. All the measurements described above were  performed at room temperature.



References  1. Branton, D. et al. The potential and challenges of nanopore sequencing. Nat. Biotech. 26, 1146–1153 (2008).  2. Venkatesan, B. M. & Bashir, R. Nanopore sensors for nucleic acid analysis. Nat. Nanotechnol. 6, 615–624 (2011).  3. Feng, Y., Zhang, Y., Ying, C., Wang, D. & Du, C. Nanopore-based Fourth-generation DNA Sequencing Technology. Genom. Proteom.  Bioinform. 13, 4–16 (2015).

 4. Deamer, D., Akeson, M. & Branton, D. Three decades of nanopore sequencing. Nat. Biotech. 34, 518–524 (2016).  5. Szalay, T. & Golovchenko, J. A. De novo sequencing and variant calling with nanopores using PoreSeq. Nat. Biotech. 33, 1087–1091  (2015).

 6. Jain, M. et al. Improved data analysis for the MinION nanopore sequencer. Nat. Methods 12, 351–356 (2015).  7. Laver, T. et al. Assessing the performance of the Oxford Nanopore Technologies MinION. Genom. Proteom. Bioinform. 3, 1–8  (2016).

 8. Goodwin, S. et al. Oxford Nanopore sequencing, hybrid error correction, and de novo assembly of a eukaryotic genome. Genome  Res. 25, 1750–1756 (2015).

 9. Ashton, P. M. et al. MinION nanopore sequencing identifies the position and structure of a bacterial antibiotic resistance island. Nat.  Biotech. 33, 296–300 (2015).

 10. Jain, M., Olsen, H. E., Paten, B. & Akeson, M. The Oxford Nanopore MinION: delivery of nanopore sequencing to the genomics  community. Genome Biol. 17, 239 (2016).

 11. Jain, M. et al. Nanopore sequencing and assembly of a human genome with ultra-long reads. Nat. Biotech. 36, 338–345 (2018).  12. Michael, T. P. et al. High contiguity Arabidopsis thaliana genome assembly with a single nanopore flow cell. Nat. Commun. 9, 541  (2018).

 13. Belser, C. et al. Chromosome-scale assemblies of plant genomes using nanopore long reads and optical maps. Nat. Plants 4, 879–887  (2018).

et al. Early MinION™ nanopore single-molecule sequencing technology enables the characterization of hepatitis B virus  genetic complexity in clinical samples. PLoS One 13(3), e0194366, https://doi.org/10.1371/journal.pone.0194366 (2018).

 15. Yusko, E. C. et al. Controlling protein translocation through nanopores with bio-inspired fluid walls. Nat. Nanotech. 6, 253–260  (2011).

 16. Larkin, J., Henley, R. Y., Muthukumar, M., Rosenstein, J. K. & Wanunu, M. High-bandwidth protein analysis using solid-state  nanopores. Biophys J. 106, 696–704 (2014).

 17. Lin, X., Ivanov, A. P. & Edel, J. B. Selective single molecule nanopore sensing of proteins using DNA aptamer-functionalised gold  nanoparticles. Chem. Sci. 8, 3905–3912 (2017).

 18. Sze, J. Y. Y., Ivanov, A. P., Cass, A. E. G. & Edel, J. B. Single molecule multiplexed nanopore protein screening in human serum using  aptamer modified DNA carriers. Nat. Commun. 8, 1552 (2017).

 19. Yu, R.-J., Ying, Y.-L., Hu, Y.-X., Gao, R. & Long, Y.-T. Label-Free Monitoring of Single Molecule Immunoreaction with a Nanopipette.  Anal. Chem. 89, 8203–8206 (2017).

 20. Morin, T. J. et al. Nanopore-Based Target Sequence Detection. PLoS One 11(5), e0154426, https://doi.org/10.1371/journal. pone.0154426 (2016).

 21. Morin, T. J. et al. A handheld platform for target protein detection and quantification using disposable nanopore strips. Sci. Rep. 8,  14834, https://doi.org/10.1038/s41598-018-33086-7 (2018).

 22. Yang, L. & Yamamoto, T. Quantification of Virus Particles Using Nanopore-Based Resistive-Pulse Sensing Techniques. Front.  Microbiol. 7, 1500 (2016).

 23. Uram, J. D., Ke, K., Hunt, A. J. & Mayer, M. Submicrometer Pore-Based Characterization and Quantification of Antibody–Virus  Interactions. Small 2, 967–972 (2006).

 24. Zhou, K., Li, L., Tan, Z., Zlotnick, A. & Jacobson, S. C. Characterization of Hepatitis B Virus Capsids by Resistive-Pulse Sensing. J.  Am. Chem. Soc. 133, 1618–1621.

 25. Arjmandi, N., Roy, W. V. & Lagae, L. Measuring Mass of Nanoparticles and Viruses in Liquids with Nanometer-Scale Pores. Anal.  Chem. 86, 4637–4641 (2014).

 26. McMullen, A., de Haan, H. W., Tang, J. X. & Stein, D. Stiff filamentous virus translocations through solid-state nanopores. Nat.  Commun. 5, 4171 (2014).

 27. Wu, H. et al. Translocation of Rigid Rod-Shaped Virus through Various Solid-State Nanopores. Anal. Chem. 88, 2502–2510 (2016).  28. Li, J. et al. Ion-beam sculpting at nanometre length scales. Nature 412, 166–169 (2001).

 29. Taniguchi, M., Tsutsui, M., Yokota, K. & Kawai, T. Fabrication of the gating nanopore device. Appl. Phys. Lett. 95, 123701 (2009).  30. Storm, A. J., Chen, J. H., Ling, X. S., Zandbergen, H. W. & Dekker, C. Fabrication of solid-state nanopores with single-nanometre  precision. Nat. Mater. 2, 537–540 (2003).

 31. van den Hout, M. et al. Controlling nanopore size, shape and stability. Nanotechnology 21, 115304 (2010).  32. Yang, J. et al. Rapid and precise scanning helium ion microscope milling of solid-state nanopores for biomolecule detection.  Nanotechnology 22, 285310 (2011).

 33. Sawafta, F., Carlsen, A. T. & Hall, A. R. Membrane Thickness Dependence of Nanopore Formation with a Focused Helium Ion Beam.  Sensors 14, 8150–8161 (2014).

 34. Carlsen, A. T., Zahid, O. K., Ruzicka, J., Taylor, E. W. & Hall, A. R. Interpreting the Conductance Blockades of DNA Translocations  through Solid-State Nanopores. ACS Nano 8, 4754–4760 (2014).

 35. Kwok, H., Briggs, K. & Tabard-Cossa, V. Nanopore Fabrication by Controlled Dielectric Breakdown. PLoS ONE 9(3), e92880,  https://doi.org/10.1371/journal.pone.0092880 (2014).



 36. Briggs, K., Kwok, H. & Tabard-Cossa, V. Automated Fabrication of 2-nm Solid-State Nanopores for Nucleic Acid Analysis. Small 10,  2077–2086 (2014).

 37. Yanagi, I., Akahori, R., Hatano, T. & Takeda, K. Fabricating nanopores with diameters of sub-1 nm to 3 nm using multilevel pulse- voltage injection. Sci. Rep. 4, 5000, https://doi.org/10.1038/srep05000 (2014).

 38. Ying, C. et al. 3D nanopore shape control by current-stimulus dielectric breakdown. Appl. Phys. Lett. 109, 063105 (2016).  39. Briggs, K. et al. Kinetics of nanopore fabrication during controlled breakdown of dielectric membranes in solution. Nanotechnology  26, 084004 (2015).

 40. Zrehen, A., Gilboaa, T. & Meller, A. Real-time visualization and sub-diffraction limit localization of nanometer-scale pore formation  by dielectric breakdown. Nanoscale 9, 16437–16445 (2017).

 41. Wang, Y., Chen, Q., Deng, T. & Liu, Z. Nanopore fabricated in pyramidal  $\mathrm{HfO}_{2}$ film by dielectric breakdown method. Appl. Phys.  Lett. 111, 143103 (2017).

 42. Kwok, H., Waugh, M., Bustamante, J., Briggs, K. & Tabard-Cossa, V. Long Passage Times of Short ssDNA Molecules through  Metallized Nanopores Fabricated by Controlled Breakdown. Adv. Funct. Mater. 24, 7745–7753 (2014).

 43. Carlsen, A. T., Briggs, K., Hall, A. R. & Tabard-Cossa, V. Solid-state nanopore localization by controlled breakdown of selectively  thinned membranes. Nanotechnology 28, 085304 (2017).

 44. Wang, Y. et al. Fabrication of multiple nanopores in a  $\mathrm{SiN_{x}}$ membrane via controlled breakdown. Sci. Rep. 8, 1234, https://doi. org/10.1038/s41598-018-19450-7 (2018).

 45. Tahvildari, R. et al. Manipulating Electrical and Fluidic Access in Integrated Nanopore-Microfluidic Arrays Using Microvalves.  Small 13, 1602601 (2017).

 46. Tahvildari, R., Beamish, E., Tabard-Cossa, V. & Godin, M. Integrating nanopore sensors within microfluidic channel arrays using  controlled breakdown. Lab Chip 15, 1407–1411 (2015).

 47. Yanagi, I., Akahori, R., Aoki, M., Harada, K. & Takeda, K. Multichannel detection of ionic currents through two nanopores  fabricated on integrated  $\mathrm{Si}_{3}\mathrm{N}_{4}$  membranes. Lab Chip 16, 3340–3350 (2016).

 48. Yanagi, I., Fujisaki, K., Hamamura, H. & Takeda, K. Thickness-dependent dielectric breakdown and nanopore creation on sub-10- nm-thick SiN membranes in solution. Jour. Appl. Phys. 121, 045301 (2017).

 49. Goto, Y., Yanagi, I., Matsui, K., Yokoi, T. & Takeda, K. Integrated solid-state nanopore platform for nanopore fabrication via  dielectric breakdown, DNA-speed deceleration and noise reduction. Sci. Rep. 6, 31324, https://doi.org/10.1038/srep31324 (2016).

 50. Akahori, R. et al. Discrimination of three types of homopolymers in single-stranded DNA with solid-state nanopores through  external control of the DNA motion. Sci. Rep. 7, 9073, https://doi.org/10.1038/s41598-017-08290-6 (2017).

 51. Kuan, A. T., Lu, B., Xie, P., Szalay, T. & Golovchenko, J. A. Electrical pulse fabrication of graphene nanopores in electrolyte solution.  Appl. Phys. Lett. 106, 203109 (2015).

 52. Feng, J. et al. Electrochemical Reaction in Single Layer MoS2: Nanopores Opened Atom by Atom. Nano Lett. 15, 3431–3438 (2015).  53. Arcadia, C. E., Reyes, C. C. & Rosenstein, J. K. In Situ Nanopore Fabrication and Single-Molecule Sensing with Microscale Liquid  Contacts. ACS Nano 11, 4907–4915 (2017).

 54. Goto, Y., Yanagi, I., Matsui, K., Yokoi, T. & Takeda, K. Identification of four single-stranded DNA homopolymers with a solid-state  nanopore in alkaline CsCl solution. Nanoscale 10(44), 20844–20850 (2018).

 55. Yamazaki, H., Hu, R., Zhao, Q. & Wanunu, M. Photothermally Assisted Thinning of Silicon Nitride Membranes for Ultrathin  Asymmetric Nanopores. ACS Nano 12, 12472–12481 (2018).

 56. Yanagi, I., Hamamura, H., Akahori, R. & Takeda, K. Two-step breakdown of a SiN membrane for nanopore fabrication: Formation  of thin portion and penetration. Sci. Rep. 8, 10129, https://doi.org/10.1038/s41598-018-28524-5 (2018).

 57. Ying, C. et al. Formation of Single Nanopores with Diameters of  $20–50\ \mathrm{nm}$  in Silicon Nitride Membranes Using Laser-Assisted  Controlled Breakdown. ACS nano 12, 11458–11470 (2018).

 58. Bello, J. & Shim, J. Solid-state nanopore fabrication in LiCl by controlled dielectric breakdown. Biomed Microdevices 20, 38 (2018).  59. Zhang, Y. et al. Nanopore Formation via Tip-Controlled Local Breakdown Using an Atomic Force Microscope. Small Methods,  https://doi.org/10.1002/smtd.201900147 (2019).



 60. Gilboa, T., Zvuloni, E., Zrehen, A., Squires, A. H. & Meller, A. Automated, Ultra-Fast Laser-Drilling of Nanometer Scale Pores and  Nanopore Arrays in Aqueous Solutions. Adv. Funct. Mater. 1900642, https://doi.org/10.1002/adfm.201900642 (2019).

 61. Roshan, K. A., Tang, Z. & Guan, W. High fidelity moving Z-score based controlled breakdown fabrication of solid-state nanopore.  Nanotechnology 30, 095502 (2019).

 62. Williams, K. R. & Muller, R. S. Etch Rates for Micromachining Processing. J. MICROELECTROMECH. S. 5, 256–269 (1996).  63. Williams, K. R., Gupta, K. & Wasilik, M. Etch rates for micromachining processing-Part II. J. Microelectromech. S. 12, 761–778  (2003).

 64. Lepoitevin, M., Ma, T., Bechelany, M., Janot, J.-M. & Balme, S. Functionalization of single solid state nanopores to mimic biological  ion channels: A review. Adv. Colloid. Interface Sci. 250, 195–213 (2017).

 65. Zeng, Z., Ai, Y. & Qian, S. pH-regulated ionic current rectification in conical nanopores functionalized with polyelectrolyte brushes.  Phys. Chem. Chem. Phys. 16, 2465–2474 (2014).

 66. Daiguji, H., Oka, Y. & Shirono, K. Nanofluidic Diode and Bipolar Transistor. Nano Lett. 5, 2274–2280 (2005).  67. Daiguji, H. Ion transport in nanofluidic channels. Chem Soc Rev. 39, 901–911 (2010).

 68. Wanunu, M. et al. Rapid electronic detection of probe-specific microRNAs using thin nanopore sensors. Nat. Nanotechnol. 5,  807–814 (2010).

 69. Hall, J. E. Access resistance of a small circular pore. J. Gen. Physiol. 66, 531–532 (1975).

 70. Larkin, J. et al. Slow DNA Transport through Nanopores in Hafnium Oxide Membranes. ACS Nano 7, 10121–10128 (2013).  71. Yanagi, I., Ishida, T., Fujisaki, K. & Takeda, K. Fabrication of 3-nm-thick  $\mathrm{Si}_{3}\mathrm{N}_{4}$ membranes for solid-state nanopores using the poly- Si sacrificial layer process. Sci. Rep. 5, 14656, https://doi.org/10.1038/srep14656 (2015).

Acknowledgements We thank Mr. Yusuke Goto, Dr. Takahide Yokoi, Dr. Yoshimitsu Yanagawa, Dr. Chihiro Uematsu and Dr.  Yoshinobu Kohara of Hitachi for their valuable discussions and critical comments on the manuscript. A part of  this work was supported by the NIMS Microstructural Characterization Platform (NMCP) as a programme of  the “Nanotechnology Platform” of the Ministry of Education, Culture, Sports, Science and Technology (MEXT),  Japan. We are grateful to Dr. Noriko Isaka at NMCP for her support regarding the operation of the TEM.

Author contributions I.Y. and R.A. developed the initial concept, designed the experiments, performed experiments and analysed the  data. K.T. supervised the study.

Additional information Supplementary information accompanies this paper at https://doi.org/10.1038/s41598-019-49622-y.

Competing Interests: Financial support for this study was provided by Hitachi, Ltd. I.Y., R.A. and K.T. are  current employees of Hitachi, Ltd. The authors declare no non-financial conflict of interests.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and  institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International  License, which permits use, sharing, adaptation, distribution and reproduction in any medium or  format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Cre- ative Commons license, and indicate if changes were made. The images or other third party material in this  article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the  material. If material is not included in the article’s Creative Commons license and your intended use is not per- mitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the  copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

   $\copyright$  The Author(s) 2019
# open

### Received: 13 June 2019

### Accepted: 28 August 2019

### Published: xx xx xxxx

# Stable fabrication of a large  nanopore by controlled dielectric  breakdown in a high-pH solution  for the detection of various-sized  molecules

itaru Yanagi, Rena Akahori & Ken-ichi takeda for nanopore sensing of various-sized molecules with high sensitivity, the size of the nanopore should  be adjusted according to the size of each target molecule. for solid-state nanopores, a simple and  inexpensive nanopore fabrication method utilizing dielectric breakdown of a membrane is widely used.  This method is suitable for fabricating a small nanopore. However, it suffers two serious problems when  attempting to fabricate a large nanopore: the generation of multiple nanopores and the non-opening  failure of a nanopore. in this study, we found that nanopore fabrication by dielectric breakdown of a  Sin membrane under high-pH conditions  $(\mathsf{p H}\geq11.3)$  could overcome these two problems and enabled  the formation of a single large nanopore up to  $40\,{\mathsf{n m}}$  in diameter within one minute. Moreover, the  ionic-current blockades derived from streptavidin-labelled and non-labelled DnA passing through the  fabricated nanopore were clearly distinguished. the current blockades caused by streptavidin-labelled  DNA could be identified even when its concentration is  $\mathbf{1}\%$ of the total DNA.



Nanopores have been widely used in recent years as a highly sensitive microscope that can observe various bio- molecules in an aqueous solution. The structural and electrical characteristics of the target molecules can be  extracted by detecting the changes in ionic currents through the nanopore when the molecules pass through it.  For instance, single-molecule DNA sequencing is currently possible–14 by using a biological nanopore with a  diameter of 2 nm or less. In addition, the detection of proteins, antigen-antibody complexes and probe-labelled  DNA is possible–21 by using a solid-state nanopore with a diameter of several to tens of nanometres. Moreover,  the detection of various viruses is also possible–27 by using a solid-state nanopore with a diameter of several tens  to hundreds nanometers. To detect such varied molecules with high sensitivity, adjusting the size of the nanopore  according to the size of each target molecule is important.

For solid-state nanopores, a nanopore with a diameter of a few to hundreds of nanometres can be fabricated  by choosing an appropriate manufacturing process–38. This capability is an advantage over biological nanopores  whose diameters are typically limited to approximately 2 nm or less. A nanopore with a diameter of approximately   $20\,\mathrm{nm}$  to several hundred nanometres can be fabricated by utilizing optical or electron-beam (EB) lithography  followed by reactive-ion etching. An even smaller nanopore down to 1 nm in diameter can be fabricated by  drilling a membrane with a focused-electron or helium-ion beam30–34. In addition, dielectric breakdown of a  membrane has been widely utilized as a method of nanopore fabrication in recent years35–61 because of its sim- plicity and inexpensiveness. With the application of a high constant voltage to a membrane and termination of the  voltage when the current between the electrodes reaches a predetermined cut-off value, a nanopore can be created  in a membrane. This method is called controlled breakdown (CBD)35,,, which also enables the fabrication of a  small nanopore down to 1 nm in diameter.

CBD, however, suffers two problems for the fabrication of a single large nanopore with a diameter of larger  than approximately 5 nm. One is the generation of multiple nanopores, which has been reported in several  studies,,,. Our observed example of the generation of multiple nanopores in a 10-nm-thick SiN membrane is  also shown in Supplementary Fig. 1. This behaviour is caused by the second and the subsequent dielectric break- downs of the membrane occurring until the current reaches the cut-off current after the first dielectric breakdown  (i.e., multiple nanopores generate while widening a nanopore generated at the first dielectric breakdown). When  trying to fabricate a large nanopore, it is necessary to set a high cut-off current value. As a result, the time spent on  the nanopore widening process increases, leading to a high risk of the generation of multiple nanopores.



This problem seems to be solved if the time of the nanopore widening process, compared to the time to the  first dielectric breakdown, can be sufficiently shortened. In this context, a thicker membrane is expected to be  suitable for the fabrication of a single large nanopore. A higher voltage is required to break down a thicker mem- brane. Consequently, a higher Joule energy is released after the first dielectric breakdown, which contributes to  faster nanopore enlargement. However, the second problem appears here. As we reported previously, when  CBD was applied to a thick (20-nm-thick) SiN membrane, a nanopore could not be fabricated, and only a local  conductive-film portion was created. With an increase in the cut-off current value, the area of the conductive-film  portion expanded, and the membrane was eventually destroyed. Therefore, the fabrication of a single large nan- opore by CBD has been very difficult.

Recently, it has been reported that the laser-assisted CBD can form a large nanopore with a diameter of  . While this technique is revolutionary, it requires a laser optical system for nanopore fabrication. In  $20–50\,\mathrm{{nm}^{57}}$  this study, we report a method that enables the fabrication of a single large nanopore in a thick membrane by CBD  without the need for optics. The application of CBD to a SiN membrane with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  in an  aqueous solution with a pH higher than 11.3 enabled the fabrication of a nanopore instead of a conductive-film  portion. The diameter of the fabricated nanopore could be roughly controlled within a range from  $5\,\mathrm{nm}$  to  $40\,\mathrm{nm}$   by changing the cut-off current value. In addition, the occurrences of streptavidin (SA)-labelled and non-labelled  DNA passing through a nanopore were clearly discriminated by monitoring the difference in ionic-current block- ades derived from those molecules. Even if the SA-labelled DNA was present in only  $1\%$  of the total DNA in an  aqueous solution, its presence could be detected.

Results The schematic setup for the dielectric breakdown experiments is presented in Fig. 1(a). The area of the SiN mem- brane with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  was restricted within approximately a small square area of approx- imately  $600\times600~\mathrm{nm}^{2}$  so that fabricated nanopores could be easily found. Two  $\mathrm{Ag/AgCl}$ electrodes (cis and  trans electrodes) were immersed in 1 M KCl aqueous solution at various  $\mathrm{pH}$  values for applying voltages and  measuring currents through the membrane. In this study, SiN membranes with thicknesses of  $20\,\mathrm{nm}$  were mainly  used for the dielectric breakdown experiments. Figure 1(b) presents examples of current-time traces when CBD  was applied to 20-nm-thick SiN membranes under ten different pH conditions. The voltage applied was 20 V  ( $V_{\mathrm{trans}}\,{=}\,0\,\mathrm{V}$  and  $V_{\mathrm{cis}}\,{=}\,20\,\mathrm{V})$ , and the cut-off current  $(I_{\mathrm{cutoff}})$  was set at  $1\,\upmu\mathrm{A}.$   $I_{\mathrm{cutoff}}$ was set as a limit of the output  current of the measuring instrument, and the voltage automatically dropped when the current between the elec- trodes reached $I_{\mathrm{cutoff}}\pm0.1\%$ . The dielectric breakdown point was clearly confirmed in each time trace. Figure 1(c)  presents the plots of the time to breakdown (TBD) under each pH condition. The TBD became shorter as the  aqueous solution became more acidic or alkaline. In particular, faster breakdowns were prominently observed  under alkaline conditions than under neutral or acidic conditions. Therefore, Si-N bond breakage in the mem- brane was thought to be strongly promoted by electrochemical reactions induced particularly by hydroxide ions.  Notably, this trend is opposite to that reported by Kwok et al.. According to their report, the TBD of SiN mem- branes at a  $\mathrm{pH}$  of 2 was approximately 100 times shorter than that at a pH of 13.5. In addition, the longest TBD  was observed at a pH of approximately 10 in their report, while it was observed at a pH of approximately 5 in our  result. These differences are thought to be due to the difference in deposition conditions of the SiN membrane  layers, which affects their basic material properties. Details of the formation process of our SiN membranes are  described in the Methods section. Incidentally, the reason for shorter TBD under acidic condition is thought to  be due to $\mathrm{H^{+}}$  or hole injection into the membrane which strongly promoted impact ionization-induced avalanche  and Si-N bond breaking,.

Figure 2 presents transmission electron microscopy (TEM) images of ten different 20-nm-thick SiN mem- branes after CBD under ten different pH conditions. During CBD, the applied voltage and  $I_{\mathrm{cutoff}}$ were set at  $20\,\mathrm{V}$  ( $V_{\mathrm{trans}}\,{=}\,0\,\mathrm{V}$  and  $V_{\mathrm{cis}}\,{=}\,20\,\mathrm{V})$  and  $1\,\upmu\mathrm{A},$  respectively. An image of the entire membrane and a magnified view of the  area around the created defect are shown in each image set. From the entire images of the membranes, the num- ber of created defects (indicated by yellow arrows) was confirmed to be one per membrane. The magnified views  of the defects confirmed that there were two kinds of cases: instead of a nanopore, a local conductive-film portion  was created (i.e., amorphous material was confirmed in the defect area), or a nanopore was created (i.e., no amor- phous material was confirmed in the defect area). The conductive-film portion is thought to be a low-density film  that allows ion conduction. We previously reported that only a conductive-film portion, not a nanopore, was cre- ated when CBD was applied to a 20-nm-thick SiN membrane at a pH of 7.556. In the present experiment, the same  result was obtained even when CBD was performed in a pH range from 1 to 11. On the other hand, interestingly, a  nanopore was created when CBD was performed in the solution with a pH higher than 11.3. In addition, the size  of the nanopore created increased as the  $\mathrm{pH}$ of the solution increased. We subjected at least three membranes to  CBD at each pH and confirmed the same trend described above. This result suggests that the membrane could be  etched by hydroxide ions in the process of dielectric breakdown. Generally, it is known that SiN is hardly etched  and Si is easily etched in an alkaline aqueous solution,. Therefore, the membrane was assumed to be etched by  the following two steps: (i) Si-N bonds were destabilized by a high electric field during CBD, and the number of Si  dangling bonds and Si-Si bonds increased. (ii) Si was etched by the following chemical reaction,:

## 
$$
\mathrm{Si\mathrm{~+~}4O H^{-}\rightarrow S i(O H)_{4}\mathrm{~+~}4e^{-}},
$$


## 
$$
2\mathrm{H}_{2}\mathrm{O}+2\mathrm{e}^{-}\rightarrow2\mathrm{OH}+\mathrm{H}_{2}.
$$


(1) (2)



Figure 3(a) presents the corresponding I-V characteristics for each membrane after CBD. Figure 3(b) presents  the pH dependence of  $I_{\mathrm{rec}}\,{=}\,\big|I(0.4\,\mathrm{V})/I(-0.4\,\mathrm{V})\big|,$  which represents the degree and direction of each  $I^{-}V$ curve’s  rectification. In the case of  $\mathrm{pH}\!\geq\!11.3$  where a nanopore was created, each  $I^{-}V$ curve showed almost ohmic char- acteristics, and  $I_{\mathrm{rec}}$  was approximately 1. On the other hand, in the case of  $\mathrm{pH}\,1$  to 11 where a nanopore was not  created, the shape of the  $I^{-}V$ curve and  $I_{\mathrm{rec}}$  varied in accordance with the  $\mathrm{pH}$  value. A convex upward I-V curve  (i.e.,  $I_{\mathrm{rec}}\!<\!1)$  at  $\mathrm{pH}\;1\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;$ , an almost ohmic  $I^{-}V$ curve (i.e.,  $I_{\mathrm{rec}}\!=\!1)$  at  $\mathrm{pH}\,3$ , and convex downward I-V curves (i.e.,   at  $\mathrm{pH}\,5\,\$  to 11 were confirmed. $I_{\mathrm{rec}}\!>\!1)$  $I_{\mathrm{rec}}$  monotonically increased with an increase in  $\mathrm{pH}$ .

This result provided us the distribution of conduction carriers (i.e., conduction ions) in the conductive-film  portion at each  $\mathrm{pH}$  without voltage biases. Generally, a rectification of an  $I^{-}V$ curve appears when there is an  asymmetric carrier distribution in the direction of current flow. Semiconductor PN junction devices are the  most well-known examples. Recently, it has been pointed out that such a rectification also occurs in a current  through a conical nanopore, or nanofluidic channel, due to the asymmetric carrier distribution. Figure 3(c)  presents the assumed conduction-ion distributions in the conductive-film portions. The density of negative con- duction ions was thought to be high on the trans side of the conductive-film portion at  $\mathrm{pH}\,1$  because  $I_{\mathrm{rec}}\!<\!1$  and   was a forward-biased state. In the case of pH 3,  $I_{\mathrm{rec}}$  $V_{\mathrm{trans}}<V_{\mathrm{cis}}$ was approximately 1, which suggested that there  was almost no asymmetry in the distribution of conduction ions. On the other hand, the density of positive  conduction ions was thought to be high on the trans side of the conductive-film portion at  $\mathrm{pH}\,5$  to 11 because   and  $I_{\mathrm{rec}}\!>\!1$  $V_{\mathrm{trans}}<V_{\mathrm{cis}}$  was a reverse-biased state. These positively and negatively biased carrier distributions were  thought to be induced because the conductive-film portion itself was negatively or positively charged depending  on the pH value; that is, the isoelectric point of the conductive-film portion was assumed to be approximately 3,  and the portion was positively or negatively charged when the pH of the solution was smaller or larger than pH 

## 
$$
d_{\mathrm{TEM}}=2\bigg(\frac{S_{\mathrm{TEM}}}{\pi}\bigg)^{1/2},
$$


(3) where  $S_{\mathrm{TEM}}$  is the area of the nanopore surrounded by the yellow line in each TEM image, as shown in Fig. 5(a).  The area was measured using image processing software (ImageJ). Figure 5(b) presents the relationship between   and $d_{\mathrm{{TEM}}}$  $I_{\mathrm{cutoff}}\left(N\!=\!24\right)$ . Nanopores with a  $d_{\mathrm{TEM}}$  of approximately 5 to  $40\,\mathrm{nm}$  could be fabricated by changing the  value of  $I_{\mathrm{cutoff}}$ from 0.1 to  $6.5\,\upmu\mathrm{A}$ . Note that the number of nanopores observed was always one per membrane.  Figure 5(c) presents the relationship between the conductance  $(G)$  of the ionic current through the nanopore  and  $d_{\mathrm{TEM}}$ . The solid line represents the theoretically calculated relation assuming a cylindrical nanopore, which  is obtained as follows–70:



## 
$$
G=\sigma\bigg(\frac{4h_{\mathrm{eff}}}{\pi d^{2}}+\frac{1}{d}\bigg)^{-1},
$$


(4) where  $\upsigma\,{=}\,0.123\,\mathrm{S/cm}$  is the measured conductance of a KCl aqueous solution with a pH of 12.7 at  $24\,^{\circ}\mathrm{C},$   $d$ is the  diameter of the nanopore, and  $h_{\mathrm{eff}}$ is the height of the nanopore (often called the effective thickness of the nanop- ore). The plotted measurements could be well fitted with the theoretically calculated line by setting  $h_{\mathrm{eff}}$ at  $12.5\,\mathrm{nm}$ .  According to many previous reports,,–71,  $h_{\mathrm{eff}}$ is smaller than the actual membrane thickness. The result in this  study is also consistent with this trend. The times required to form the nanopores by CBD were less than 1 min  regardless of the size of the nanopore (see Supplementary Fig.  $4(\mathrm{a}).$ ).



CBD under high-pH conditions was also effective for nanopore fabrication in 14-nm-thick SiN membranes  (Fig. 6). The current-time traces during CBD and images of the membrane after CBD for each case at  $\mathrm{pH}\,7.5$ and  12.7 are presented in Fig. 6(a,b). Similar to the case of the  $20\mathrm{-nm}$ -thick SiN membranes, CBD in a 14-nm-thick  SiN membrane provided a conductive-thin portion instead of a nanopore at  $\mathrm{pH}\,7.5$ , whereas a nanopore was cre- ated in the case of  $\mathrm{pH}\,12.7\,\$ . Figure  $6(c)$  presents the dependence of  $d_{\mathrm{TEM}}$ on  $I_{\mathrm{cutoff}}\left(N\!=\!6\right)$ . All the nanopores were  fabricated by CBD at an applied voltage of 11 V and  $\mathrm{pH}$  of 12.7, and the  $d_{\mathrm{{TEM}}}$ became larger with an increase in  . Figure  $I_{\mathrm{cutoff}}.$ 6(d) presents the relationship between  $G$  and  $d_{\mathrm{{TEM}}}$ . The effective thickness  $(h_{\mathrm{eff}})$  was estimated to be  To identify the current-blockade signals derived from SA-labelled dsDNA, free unbound SA should not pass  through the nanopore. Therefore, we first examined whether free SA can pass through the nanopore. Figure  $^{7(\mathrm{b},\mathrm{c})}$   present respectively the time traces of ionic currents through the nanopores fabricated in the 20-nm- and  14-nm-thick SiN membranes when the aqueous solution in the cis chamber contained  $100\,\mathrm{nM}$ SA. Almost no  ionic-current blockades derived from SA translocation through the nanopores were confirmed (approximately  less than 5 events/min). In the following experiments, the concentration of free SA in aqueous solution did not  exceed  $100\,\mathrm{nM}$ . Figure 7(d) presents the ionic current through the nanopore fabricated in the 20-nm-thick  SiN membrane when the aqueous solution in the cis chamber contained 15 nM 400-bp dsDNA. Numerous  ionic-current blockades caused by dsDNA translocations through the nanopore were observed. After the meas- urement, the solution in the cis chamber was partly displaced by the solution of 1 M KCl with  $15\,\mathrm{nM}$ SA-labelled  80-bp dsDNA so that SA-labelled and non-labelled dsDNA could be mixed in the cis chamber. Figure 7(e) pre- sents the ionic current through the nanopore after SA-labelled and non-labelled DNA were mixed in the cis  chamber. Larger current blockades than those derived from dsDNA were frequently observed, which indicated  the passage of SA-labelled dsDNA through the nanopore.



Magnified views of the typical current blockades when SA-labelled and non-labelled dsDNA passed through  the nanopore are presented in Fig. 7(f). The dwell times of the current blockades derived from SA-labelled 80-bp  dsDNA were typically shorter than those derived from 400-bp dsDNA, reflecting the difference in the dsDNA  lengths. The scatter plots of the dwell time and  $\Delta I_{\mathrm{max}}$ and the corresponding histograms of before and  $\Delta I_{\mathrm{max}}$  after the mixture of SA-labelled and non-labelled dsDNA are presented in Fig.  $7(\mathrm{g-j})$ .  $\Delta I_{\mathrm{max}}$  was determined as  the maximum current-blockade value observed in each current-blockade event. The figures confirmed that the  occurrences of SA-labelled and non-labelled dsDNA passing through the nanopore could be clearly distinguished  based on the value of  $\Delta I_{\mathrm{max}}$ .

Next, we examined the event discrimination of SA-labelled and non-labelled 400-bp dsDNA passing through  nanopores fabricated in 14-nm-thick SiN membranes (Fig. 8). Nine different aqueous solutions for the cis cham- ber were prepared; that is, a solution of 1 M KCl with  $15\,\mathrm{nM}$  SA-labelled 400-bp dsDNA and a solution of 1 M KCl  with  $15\,\mathrm{nM}\,400.$ -bp dsDNA were mixed at ratios of 1:0, 9:1, 7:3. 5:5, 3:7, 1:9, 5:95, 1:99 and 0:1. Figure 8(a) pre- sents five results from the nine experiments. Compared to the results shown in Fig. 7(h,j), these scatter plots and   histograms exhibit less clear boundaries between the two events derived from SA-labelled and non-labelled  $\Delta I_{\mathrm{max}}$  dsDNA. We cannot currently provide a clear explanation for this difference. Possible causes include difference in  the thicknesses of the SiN membranes ( $20\,\mathrm{nm}$ or  $14\,\mathrm{nm}$ ) and lengths of dsDNA (80 bp or 400 bp). We provision- ally determined the boundary value of  $\Delta I_{\mathrm{max}}$ derived from SA-labelled and non-labelled dsDNA as  $c+4w,$  where   is the central value of a Gaussian curve fitted to each spectrum around  $c$  $\Delta I_{\mathrm{max}}\,{=}\,0.5\,\mathrm{nA}$ , which was attributed to  the events mostly caused by dsDNA translocation through the nanopore, and  $w$ is the value of the full width at  half maximum (FWHM) of the Gaussian curve.

Small current blockades derived from dsDNA translocation through the nanopore were observed even in the  case of SA-labelled dsDNA:  $\mathrm{{dsDNA}}\,{=}\,1{:}\,0$ due to the remaining dsDNA unreacted with SA. On the other hand,  large current blockades derived from SA-labelled dsDNA translocation through the nanopore could be clearly  observed even in the case of SA-labelled dsDNA:  $\mathrm{dsDNA}\,{=}\,1{:}99\$ . Figure 8(b) presents magnified views of the typ- ical current blockades derived from SA-labelled 400-bp dsDNA. Each waveform consists of two current-blockade  levels, which were not observed when SA-labelled 80-bp dsDNA passed through the nanopore (Fig. 7(f)). Each  of the two levels is believed to be attributable to the partial entry of dsDNA or SA into the nanopore. Figure 8(c)  presents the frequency ratio of the current-blockade events derived from SA-labelled and non-labelled dsDNA in  each of the nine experiments. The figure confirms that the event frequency ratio changed, reflecting the mixing  ratio of the two solutions containing SA-labelled dsDNA and dsDNA.

Discussion Stable fabrication of a single large nanopore in a SiN membrane by CBD was examined. First, we applied CBD to  20-nm-thick SiN membranes under various pH conditions. Instead of a nanopore, a local conductive-film por- tion was created in the membrane in the case of  $\mathrm{pH}<11.3$ . On the other hand, a single nanopore was fabricated  when  $\mathrm{pH}\geq11.3$ . This result suggests that the membrane could be etched to form a nanopore by hydroxide ions  under a high electric field. The conduction-ion distribution in the conductive-film portion, incidentally, could  be extrapolated from the shape of the I-V curve, which changed depending on the pH of the aqueous solution.

The size of the nanopore could be controlled by changing the applied voltage or  $I_{\mathrm{cutoff}}$ during CBD at a  $\mathrm{pH}$  of  12.7. For example, nanopores with diameters of 5 to  $40\,\mathrm{nm}$  could be fabricated by changing the value of  $I_{\mathrm{cutoff}}$ from   to  $6.5\,\upmu\mathrm{A}$  $0.1\,\upmu\mathrm{A}$  at an applied voltage of 18 V. The effective thickness of the fabricated nanopores was found to be  approximately  $12.5\,\mathrm{nm}$ , which was estimated from the relationship between the conductance  $(G)$  and equivalent  diameter  $(d_{\mathrm{{TEM}}})$  of the nanopores. In addition, this high-pH CBD method was also effective in nanopore fabri- cation in 14-nm-thick SiN membranes. Nanopores with diameters of 7 to  $12\,\mathrm{nm}$  could be fabricated by changing  from  $0.3\,\upmu\mathrm{A}$  $I_{\mathrm{cutoff}}$  to  $1\,\upmu\mathrm{A}$  at an applied voltage of 11 V, and the effective thickness of their nanopores was found  to be approximately  $11\,\mathrm{nm}$ .

The occurrences of SA-labelled and non-labelled dsDNA passing through the fabricated nanopores could  be distinguished by the difference in ionic-current blockades caused by those two molecules. The ionic-current  blockades derived from SA-labelled dsDNA translocation through the nanopore could be clearly detected even  when the concentration of SA-labelled dsDNA was only  $1\%$  of the total concentration of dsDNA. In addition, the  frequency ratio of the current-blockade events caused by SA-labelled and non-labelled dsDNA changed, reflect- ing the mixing ratio of SA-labelled dsDNA and dsDNA.

The method of CBD under high-pH conditions examined in this study is a simple and rapid (less than 1 min)  nanopore fabrication method that enables the stable fabrication of various-sized solid-state nanopores without  the need for expensive equipment such as EB lithography, TEM or helium-ion microscopy (HIM). In addition,  the fabricated nanopores showed good performance in the detection of different-sized molecules. Consequently,  we believe that this nanopore fabrication method will accelerate the research and development of nanopore  devices for detecting various-sized molecules.



Methods fabrication of membranes. The SiN membranes were fabricated on an 8-inch silicon wafer with a thick- ness of  $725\,\upmu\mathrm{m}$ . The fabrication process was the same as that described in our previous report. First, a SiN  layer with a thickness of  $20\,\mathrm{nm}$  or  $14\,\mathrm{nm}$  was deposited on both sides of the Si wafer via low-pressure chemical  vapour deposition (reacting gases:  $\mathrm{SiH}_{2}\mathrm{Cl}_{2}$ -NH3; flow ratio:  $\mathrm{SiH}_{2}\mathrm{Cl}_{2}{:}\mathrm{NH}_{3}{=}1{:}25$ ;  $770\,^{\circ}\mathrm{C};$  0.5 Torr). The compo- sition ratio of the SiN layer analysed by X-ray photoelectron spectroscopy ( $\mathrm{PHI}\,5000$ VersaProbe II, X-ray: Al   ULVAC-PHI, Inc.) was approximately  $\mathrm{Si}{\cdot}\mathrm{N}\,{=}\,1{:}1{.}24\,\$  $\operatorname{K}_{\mathbb{\alpha},\mathbb{\beta}}$ . Then, a  $\mathrm{SiO}_{2}$  sacrificial layer with a thickness of approx- imately  $260\,\mathrm{nm}$  was deposited on the SiN layer on the front side of the wafer, and a SiN layer with a thickness of  approximately  $90\,\mathrm{nm}$  was deposited on both sides of the wafer. Next, the top SiN layer in each square area approx- imately  $600\times600\,\mathrm{nm}^{2}$  in size and the backside SiN layer in each corresponding  $1038\times1038\,\upmu\mathrm{m}^{2}$  square area were  subsequently etched by reactive-ion etching, followed by etching of the Si substrate with tetramethylammonium  hydroxide (TMAH) at  $85\,^{\circ}\mathrm{C}$ for approximately 9 hours. After etching of the Si substrate, the wafer was diced into  chips. Finally, the  $\mathrm{SiO}_{2}$  sacrificial layer was etched with potassium hydroxide ( $33\,\mathrm{wt.}\%$  solution of KOH at approx- imately  $70\,^{\circ}\mathrm{C}_{\lambda}$ ) before the dielectric breakdown experiment of the SiN membrane.

observation of Sin membranes. Observations of the SiN membranes were performed using a  field-emission TEM (JEM-2100F (HRP),  $200\,\mathrm{kV},$  JEOL, Ltd.). Before the observations, the membranes were  immersed in deionized water for a day to remove any salt residues. The areas of the fabricated nanopores were  measured using image processing software (ImageJ, National Institutes of Health, Bethesda, MD, USA). The  observation process described above was the same as that described in our previous report.

Setup for dielectric breakdown experiments. Initially, the SiN membrane was mounted onto a  custom-built acrylic flow cell. Two chambers (each with a volume of  $90\,\upmu\mathrm{L}$ ) separated by the membrane were  formed in the flow cell: a cis chamber and a trans chamber. Aqueous solutions at various pH values were prepared  by adding KOH or HCl to 1 M potassium chloride,  $10\,\mathrm{mM}$  Tris-HCl, and 1 mM EDTA. The pH of the aqueous  solution was measured using a LAQUA act pH/OPR/COND METER D-74 (HORIBA Scientific, Japan). After  filling the aqueous solution into both chambers, two  $\mathrm{Ag/AgCl}$ electrodes were immersed in both solutions to  ensure electrical contact. Application of a constant voltage, measurement of the current through the membrane  during CBD, and measurement of the I-V characteristics were performed using a 4156B precision semiconductor  parameter analyser (Agilent Technologies, Inc.). The cut-off current for CBD was set as the current compliance  value of the 4156B system.

Preparation of streptavidin-labelled 80-bp dsDNA for measurements. First, an 80-mer single-  stranded DNA (ssDNA) whose  $5^{\prime}$  end was modified with biotin was annealed with a primer. Both the biotin-modified   ssDNA and primer were purchased from Sigma Aldrich, Japan. Then, the primer was extended by an elongation   reaction, and biotin-modified 80-bp dsDNA was created. The sequences of the ssDNA and the primerwere   -ACGAATTCGAGCTCGGTACCCGGGGATCCTCTAGAGTCGACCTGCAGGCATGCAAGCTTGGCA  $5^{\prime}$  CTGGCCGTCGTTTTACA- $3^{\prime}$  and  $5^{\prime}.$ -TGTAAAACGACGGCCAGT- $3^{\prime}$ , respectively. The elongation reaction was   performed in  $50–\upmu\mathrm{L}$  reaction volumes consisting of  $3\,\upmu\mathrm{M}$  primer,  $3\,\upmu\mathrm{M}$  ssDNA,  $320\,\mathrm{U}$  Bst 3.0 DNA polymerase, 1X   isothermal amplification buffer, 1 mM dNTPs and  $1\,\mathrm{{mM}\,\mathrm{{MgSO}_{4}}}$ (New England Biolabs, Ipswich, MA). The time and   temperature of the reaction were 60 min and $65^{\circ}\mathrm{C},$  respectively. The product after the elongation reaction was purified   using NucleoSpin® Gel and PCR Clean-up (Takara Bio Inc, Japan). The purified product was then quantitated with a   Qubit fluorometric system (Life Technologies). A biotin-streptavidin binding reaction was performed in  $18.3\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $1.5\,\upmu\mathrm{M}$  biotin-modified dsDNA and  $10\,\upmu\mathrm{M}$  streptavidin (Thermo Fisher Scientific,  Waltham, MA). The time and temperature of the binding reaction were  $37\,^{\circ}\mathrm{C}$  and  $30\,\mathrm{min}$ , respectively.

Preparation of streptavidin-labelled 400-bp dsDNA for measurements. A portion of λDNA  (NIPPON GENE CO., LTD, Japan) was duplicated by PCR amplification using a biotin-modified forward  primer and a biotin-unmodified reverse primer to obtain biotin-modified 400-bp dsDNA. PCR amplification  was performed using MiniAmp Plus Thermal Cyclers (Thermo Fisher Scientific, Waltham, MA). Both ssDNA  and primer were purchased from Sigma Aldrich, Japan. The sequences of the forward and reverse primers  were  $5^{\prime}$ -TGCAACGAACAGGTCACTATCA- $3^{\prime}$  and  $5^{\prime}$ -GAGCAAAGCAAAACAGGCGTA- $3^{\prime}$ , respectively.  PCR amplification was performed in  $50\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $0.3~{\upmu\mathrm{M}}$  forward primer,  $0.3~{\upmu\mathrm{M}}$  reverse primer,  $50\,\mathrm{ng}$ λDNA, 1 U KOD-Plus- Neo, 1X PCR buffer for KOD -Plus- Neo,  $0.2\,\mathrm{mM}$  dNTPs, and   $1.5\,\mathrm{mM}$   $\mathrm{MgSO}_{4}$ (TOYOBO CO., LTD., Japan). The cycling parameters of PCR were  $94\,^{\circ}\mathrm{C}$ for  $2\,\mathrm{min}\longrightarrow\left[98\,^{\circ}\mathrm{C}\right.$  for  $10\,\mathrm{sec}\!\rightarrow\!58\,^{\circ}\mathrm{C}$  for  $30\,\mathrm{sec}\,{\longrightarrow}\,68\,^{\circ}\mathrm{C}$  for  $30\,\mathrm{sec}]\times35$  times. The product obtained after PCR was purified using  NucleoSpin® Gel and PCR Clean-up (Takara Bio Inc., Japan). The purified product was then quantitated with a  Qubit fluorometric system (Life Technologies). A biotin-streptavidin binding reaction was performed in  $18.3\mathrm{-}\upmu\mathrm{L}$  reaction volumes consisting of  $1.5\,\upmu\mathrm{M}$  biotin-modified dsDNA and  $10\,\upmu\mathrm{M}$  streptavidin. The time and tempera- ture of the binding reaction were  $37\,^{\circ}\mathrm{C}$  and  $30\,\mathrm{min}$ , respectively.

Setup for measuring DnA translocation through a nanopore. Prior to the measurements of DNA  translocation through a nanopore, the aqueous solutions in both chambers were displaced by a 1 M KCl aqueous  solution at a  $\mathrm{pH}$  of 8.7. After that, the solution in the cis chamber was displaced by a  $1\,\mathrm{M}\;\mathrm{KCl}$ aqueous solution  containing samples (i.e., SA-labelled dsDNA and dsDNA) at a  $\mathrm{pH}$  of 8.7. The ionic-current measurements were  performed using a patch-clamp amplifier (Axopatch 200B, Axon Instruments, Union City, CA). The detected  current was low-pass-filtered with a cut-off frequency of  $10\,\mathrm{kHz}$  using a four-pole Bessel filter and then digi- tized with an NI USB-6281 18-bit DAQ AD converter (National Instruments, Austin, TX) at  $50\,\mathrm{kHz}$ . Finally,  the current was recorded on the hard disk of a personal computer. All the measurements described above were  performed at room temperature.



References  1. Branton, D. et al. The potential and challenges of nanopore sequencing. Nat. Biotech. 26, 1146–1153 (2008).  2. Venkatesan, B. M. & Bashir, R. Nanopore sensors for nucleic acid analysis. Nat. Nanotechnol. 6, 615–624 (2011).  3. Feng, Y., Zhang, Y., Ying, C., Wang, D. & Du, C. Nanopore-based Fourth-generation DNA Sequencing Technology. Genom. Proteom.  Bioinform. 13, 4–16 (2015).

 4. Deamer, D., Akeson, M. & Branton, D. Three decades of nanopore sequencing. Nat. Biotech. 34, 518–524 (2016).  5. Szalay, T. & Golovchenko, J. A. De novo sequencing and variant calling with nanopores using PoreSeq. Nat. Biotech. 33, 1087–1091  (2015).

 6. Jain, M. et al. Improved data analysis for the MinION nanopore sequencer. Nat. Methods 12, 351–356 (2015).  7. Laver, T. et al. Assessing the performance of the Oxford Nanopore Technologies MinION. Genom. Proteom. Bioinform. 3, 1–8  (2016).

 8. Goodwin, S. et al. Oxford Nanopore sequencing, hybrid error correction, and de novo assembly of a eukaryotic genome. Genome  Res. 25, 1750–1756 (2015).

 9. Ashton, P. M. et al. MinION nanopore sequencing identifies the position and structure of a bacterial antibiotic resistance island. Nat.  Biotech. 33, 296–300 (2015).

 10. Jain, M., Olsen, H. E., Paten, B. & Akeson, M. The Oxford Nanopore MinION: delivery of nanopore sequencing to the genomics  community. Genome Biol. 17, 239 (2016).

 11. Jain, M. et al. Nanopore sequencing and assembly of a human genome with ultra-long reads. Nat. Biotech. 36, 338–345 (2018).  12. Michael, T. P. et al. High contiguity Arabidopsis thaliana genome assembly with a single nanopore flow cell. Nat. Commun. 9, 541  (2018).

 13. Belser, C. et al. Chromosome-scale assemblies of plant genomes using nanopore long reads and optical maps. Nat. Plants 4, 879–887  (2018).

et al. Early MinION™ nanopore single-molecule sequencing technology enables the characterization of hepatitis B virus  genetic complexity in clinical samples. PLoS One 13(3), e0194366, https://doi.org/10.1371/journal.pone.0194366 (2018).

 15. Yusko, E. C. et al. Controlling protein translocation through nanopores with bio-inspired fluid walls. Nat. Nanotech. 6, 253–260  (2011).

 16. Larkin, J., Henley, R. Y., Muthukumar, M., Rosenstein, J. K. & Wanunu, M. High-bandwidth protein analysis using solid-state  nanopores. Biophys J. 106, 696–704 (2014).

 17. Lin, X., Ivanov, A. P. & Edel, J. B. Selective single molecule nanopore sensing of proteins using DNA aptamer-functionalised gold  nanoparticles. Chem. Sci. 8, 3905–3912 (2017).

 18. Sze, J. Y. Y., Ivanov, A. P., Cass, A. E. G. & Edel, J. B. Single molecule multiplexed nanopore protein screening in human serum using  aptamer modified DNA carriers. Nat. Commun. 8, 1552 (2017).

 19. Yu, R.-J., Ying, Y.-L., Hu, Y.-X., Gao, R. & Long, Y.-T. Label-Free Monitoring of Single Molecule Immunoreaction with a Nanopipette.  Anal. Chem. 89, 8203–8206 (2017).

 20. Morin, T. J. et al. Nanopore-Based Target Sequence Detection. PLoS One 11(5), e0154426, https://doi.org/10.1371/journal. pone.0154426 (2016).

 21. Morin, T. J. et al. A handheld platform for target protein detection and quantification using disposable nanopore strips. Sci. Rep. 8,  14834, https://doi.org/10.1038/s41598-018-33086-7 (2018).

 22. Yang, L. & Yamamoto, T. Quantification of Virus Particles Using Nanopore-Based Resistive-Pulse Sensing Techniques. Front.  Microbiol. 7, 1500 (2016).

 23. Uram, J. D., Ke, K., Hunt, A. J. & Mayer, M. Submicrometer Pore-Based Characterization and Quantification of Antibody–Virus  Interactions. Small 2, 967–972 (2006).

 24. Zhou, K., Li, L., Tan, Z., Zlotnick, A. & Jacobson, S. C. Characterization of Hepatitis B Virus Capsids by Resistive-Pulse Sensing. J.  Am. Chem. Soc. 133, 1618–1621.

 25. Arjmandi, N., Roy, W. V. & Lagae, L. Measuring Mass of Nanoparticles and Viruses in Liquids with Nanometer-Scale Pores. Anal.  Chem. 86, 4637–4641 (2014).

 26. McMullen, A., de Haan, H. W., Tang, J. X. & Stein, D. Stiff filamentous virus translocations through solid-state nanopores. Nat.  Commun. 5, 4171 (2014).

 27. Wu, H. et al. Translocation of Rigid Rod-Shaped Virus through Various Solid-State Nanopores. Anal. Chem. 88, 2502–2510 (2016).  28. Li, J. et al. Ion-beam sculpting at nanometre length scales. Nature 412, 166–169 (2001).

 29. Taniguchi, M., Tsutsui, M., Yokota, K. & Kawai, T. Fabrication of the gating nanopore device. Appl. Phys. Lett. 95, 123701 (2009).  30. Storm, A. J., Chen, J. H., Ling, X. S., Zandbergen, H. W. & Dekker, C. Fabrication of solid-state nanopores with single-nanometre  precision. Nat. Mater. 2, 537–540 (2003).

 31. van den Hout, M. et al. Controlling nanopore size, shape and stability. Nanotechnology 21, 115304 (2010).  32. Yang, J. et al. Rapid and precise scanning helium ion microscope milling of solid-state nanopores for biomolecule detection.  Nanotechnology 22, 285310 (2011).

 33. Sawafta, F., Carlsen, A. T. & Hall, A. R. Membrane Thickness Dependence of Nanopore Formation with a Focused Helium Ion Beam.  Sensors 14, 8150–8161 (2014).

 34. Carlsen, A. T., Zahid, O. K., Ruzicka, J., Taylor, E. W. & Hall, A. R. Interpreting the Conductance Blockades of DNA Translocations  through Solid-State Nanopores. ACS Nano 8, 4754–4760 (2014).

 35. Kwok, H., Briggs, K. & Tabard-Cossa, V. Nanopore Fabrication by Controlled Dielectric Breakdown. PLoS ONE 9(3), e92880,  https://doi.org/10.1371/journal.pone.0092880 (2014).



 36. Briggs, K., Kwok, H. & Tabard-Cossa, V. Automated Fabrication of 2-nm Solid-State Nanopores for Nucleic Acid Analysis. Small 10,  2077–2086 (2014).

 37. Yanagi, I., Akahori, R., Hatano, T. & Takeda, K. Fabricating nanopores with diameters of sub-1 nm to 3 nm using multilevel pulse- voltage injection. Sci. Rep. 4, 5000, https://doi.org/10.1038/srep05000 (2014).

 38. Ying, C. et al. 3D nanopore shape control by current-stimulus dielectric breakdown. Appl. Phys. Lett. 109, 063105 (2016).  39. Briggs, K. et al. Kinetics of nanopore fabrication during controlled breakdown of dielectric membranes in solution. Nanotechnology  26, 084004 (2015).

 40. Zrehen, A., Gilboaa, T. & Meller, A. Real-time visualization and sub-diffraction limit localization of nanometer-scale pore formation  by dielectric breakdown. Nanoscale 9, 16437–16445 (2017).

 41. Wang, Y., Chen, Q., Deng, T. & Liu, Z. Nanopore fabricated in pyramidal  $\mathrm{HfO}_{2}$ film by dielectric breakdown method. Appl. Phys.  Lett. 111, 143103 (2017).

 42. Kwok, H., Waugh, M., Bustamante, J., Briggs, K. & Tabard-Cossa, V. Long Passage Times of Short ssDNA Molecules through  Metallized Nanopores Fabricated by Controlled Breakdown. Adv. Funct. Mater. 24, 7745–7753 (2014).

 43. Carlsen, A. T., Briggs, K., Hall, A. R. & Tabard-Cossa, V. Solid-state nanopore localization by controlled breakdown of selectively  thinned membranes. Nanotechnology 28, 085304 (2017).

 44. Wang, Y. et al. Fabrication of multiple nanopores in a  $\mathrm{SiN_{x}}$ membrane via controlled breakdown. Sci. Rep. 8, 1234, https://doi. org/10.1038/s41598-018-19450-7 (2018).

 45. Tahvildari, R. et al. Manipulating Electrical and Fluidic Access in Integrated Nanopore-Microfluidic Arrays Using Microvalves.  Small 13, 1602601 (2017).

 46. Tahvildari, R., Beamish, E., Tabard-Cossa, V. & Godin, M. Integrating nanopore sensors within microfluidic channel arrays using  controlled breakdown. Lab Chip 15, 1407–1411 (2015).

 47. Yanagi, I., Akahori, R., Aoki, M., Harada, K. & Takeda, K. Multichannel detection of ionic currents through two nanopores  fabricated on integrated  $\mathrm{Si}_{3}\mathrm{N}_{4}$  membranes. Lab Chip 16, 3340–3350 (2016).

 48. Yanagi, I., Fujisaki, K., Hamamura, H. & Takeda, K. Thickness-dependent dielectric breakdown and nanopore creation on sub-10- nm-thick SiN membranes in solution. Jour. Appl. Phys. 121, 045301 (2017).

 49. Goto, Y., Yanagi, I., Matsui, K., Yokoi, T. & Takeda, K. Integrated solid-state nanopore platform for nanopore fabrication via  dielectric breakdown, DNA-speed deceleration and noise reduction. Sci. Rep. 6, 31324, https://doi.org/10.1038/srep31324 (2016).

 50. Akahori, R. et al. Discrimination of three types of homopolymers in single-stranded DNA with solid-state nanopores through  external control of the DNA motion. Sci. Rep. 7, 9073, https://doi.org/10.1038/s41598-017-08290-6 (2017).

 51. Kuan, A. T., Lu, B., Xie, P., Szalay, T. & Golovchenko, J. A. Electrical pulse fabrication of graphene nanopores in electrolyte solution.  Appl. Phys. Lett. 106, 203109 (2015).

 52. Feng, J. et al. Electrochemical Reaction in Single Layer MoS2: Nanopores Opened Atom by Atom. Nano Lett. 15, 3431–3438 (2015).  53. Arcadia, C. E., Reyes, C. C. & Rosenstein, J. K. In Situ Nanopore Fabrication and Single-Molecule Sensing with Microscale Liquid  Contacts. ACS Nano 11, 4907–4915 (2017).

 54. Goto, Y., Yanagi, I., Matsui, K., Yokoi, T. & Takeda, K. Identification of four single-stranded DNA homopolymers with a solid-state  nanopore in alkaline CsCl solution. Nanoscale 10(44), 20844–20850 (2018).

 55. Yamazaki, H., Hu, R., Zhao, Q. & Wanunu, M. Photothermally Assisted Thinning of Silicon Nitride Membranes for Ultrathin  Asymmetric Nanopores. ACS Nano 12, 12472–12481 (2018).

 56. Yanagi, I., Hamamura, H., Akahori, R. & Takeda, K. Two-step breakdown of a SiN membrane for nanopore fabrication: Formation  of thin portion and penetration. Sci. Rep. 8, 10129, https://doi.org/10.1038/s41598-018-28524-5 (2018).

 57. Ying, C. et al. Formation of Single Nanopores with Diameters of  $20–50\ \mathrm{nm}$  in Silicon Nitride Membranes Using Laser-Assisted  Controlled Breakdown. ACS nano 12, 11458–11470 (2018).

 58. Bello, J. & Shim, J. Solid-state nanopore fabrication in LiCl by controlled dielectric breakdown. Biomed Microdevices 20, 38 (2018).  59. Zhang, Y. et al. Nanopore Formation via Tip-Controlled Local Breakdown Using an Atomic Force Microscope. Small Methods,  https://doi.org/10.1002/smtd.201900147 (2019).



 60. Gilboa, T., Zvuloni, E., Zrehen, A., Squires, A. H. & Meller, A. Automated, Ultra-Fast Laser-Drilling of Nanometer Scale Pores and  Nanopore Arrays in Aqueous Solutions. Adv. Funct. Mater. 1900642, https://doi.org/10.1002/adfm.201900642 (2019).

 61. Roshan, K. A., Tang, Z. & Guan, W. High fidelity moving Z-score based controlled breakdown fabrication of solid-state nanopore.  Nanotechnology 30, 095502 (2019).

 62. Williams, K. R. & Muller, R. S. Etch Rates for Micromachining Processing. J. MICROELECTROMECH. S. 5, 256–269 (1996).  63. Williams, K. R., Gupta, K. & Wasilik, M. Etch rates for micromachining processing-Part II. J. Microelectromech. S. 12, 761–778  (2003).

 64. Lepoitevin, M., Ma, T., Bechelany, M., Janot, J.-M. & Balme, S. Functionalization of single solid state nanopores to mimic biological  ion channels: A review. Adv. Colloid. Interface Sci. 250, 195–213 (2017).

 65. Zeng, Z., Ai, Y. & Qian, S. pH-regulated ionic current rectification in conical nanopores functionalized with polyelectrolyte brushes.  Phys. Chem. Chem. Phys. 16, 2465–2474 (2014).

 66. Daiguji, H., Oka, Y. & Shirono, K. Nanofluidic Diode and Bipolar Transistor. Nano Lett. 5, 2274–2280 (2005).  67. Daiguji, H. Ion transport in nanofluidic channels. Chem Soc Rev. 39, 901–911 (2010).

 68. Wanunu, M. et al. Rapid electronic detection of probe-specific microRNAs using thin nanopore sensors. Nat. Nanotechnol. 5,  807–814 (2010).

 69. Hall, J. E. Access resistance of a small circular pore. J. Gen. Physiol. 66, 531–532 (1975).

 70. Larkin, J. et al. Slow DNA Transport through Nanopores in Hafnium Oxide Membranes. ACS Nano 7, 10121–10128 (2013).  71. Yanagi, I., Ishida, T., Fujisaki, K. & Takeda, K. Fabrication of 3-nm-thick  $\mathrm{Si}_{3}\mathrm{N}_{4}$ membranes for solid-state nanopores using the poly- Si sacrificial layer process. Sci. Rep. 5, 14656, https://doi.org/10.1038/srep14656 (2015).

Acknowledgements We thank Mr. Yusuke Goto, Dr. Takahide Yokoi, Dr. Yoshimitsu Yanagawa, Dr. Chihiro Uematsu and Dr.  Yoshinobu Kohara of Hitachi for their valuable discussions and critical comments on the manuscript. A part of  this work was supported by the NIMS Microstructural Characterization Platform (NMCP) as a programme of  the “Nanotechnology Platform” of the Ministry of Education, Culture, Sports, Science and Technology (MEXT),  Japan. We are grateful to Dr. Noriko Isaka at NMCP for her support regarding the operation of the TEM.

Author contributions I.Y. and R.A. developed the initial concept, designed the experiments, performed experiments and analysed the  data. K.T. supervised the study.

Additional information Supplementary information accompanies this paper at https://doi.org/10.1038/s41598-019-49622-y.

Competing Interests: Financial support for this study was provided by Hitachi, Ltd. I.Y., R.A. and K.T. are  current employees of Hitachi, Ltd. The authors declare no non-financial conflict of interests.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and  institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International  License, which permits use, sharing, adaptation, distribution and reproduction in any medium or  format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Cre- ative Commons license, and indicate if changes were made. The images or other third party material in this  article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the  material. If material is not included in the article’s Creative Commons license and your intended use is not per- mitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the  copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

   $\copyright$  The Author(s) 2019
Intracellular calcium levels and enzyme secretion in response to tobacco vs ethanol in pancreatic acinar cells M. Luaces-Regueira , M. Castiñeira-Alvariño , M. Castro-Manzanares 2, A. Carrascal-Miniño , M. Campos-Toimil, J.E. Domínguez-Muñoz1.



 Foundation for Research in Digestive Diseases, Department of Gastroenterology, University Hospital of Santiago de Compostela, Spain  Farmacology Department, University of Santiago de Compostela, Spain Alterations in intracellular calcium levels and enzyme secretion in acinar cells are involved in the pathophysiology of chronic pancreatitis (CP). It has been previously shown that alcohol induces an increase of intracellular calcium, but its effect in enzyme secretion is controversial. The effect of tobacco in this context is unknown.



Hypothesis: Tobacco, similarly to alcohol, promotes the development of CP by altering intracellular calcium levels and enzyme secretion in acinar cells.

Aim: To evaluate the effect of tobacco compared with alcohol in intracellular calcium levels and amylase secretion in pancreatic acinar cells.

Methods: Experimental in-vitro study on primary acinar cell culture. Acinar cells were isolated by enzymatic and mechanic degradation from Swiss mice pancreas. Cells were stimulated by increasing concentrations of alcohol ( $.10{-}100\ \mathrm{mM})$  and tobacco  $(0.1{-}0.5~\mathrm{mg/ml})$ , and with CCK as posi- tive control. Intracellular calcium levels were measured by ﬂuorescence andamylasesecretionbyusing $\mathfrak{p}$ -nitrophenyl-maltohexaosideas substrate. Data were analysed by ANOVA.

Results: Tobacco induced a dose-dependent increase of intracellular calcium levels from  $11.38{\pm}4.89\%\left(0.1\;\mathrm{mg/ml}\right)$  to  $56.26{\pm}13.14\%\left(0.5\;\mathrm{mg/ml}\right)$ .  ( $10\,\mathrm{\mM})$  to $[14.40\pm2.06\%$   $59.8{\pm}2.57\%\left(75\mathrm{mM}\right)]$  with tobacco (up to  $21\%$  at  $0.4\;\mathrm{mg/ml})$ , but not with alcohol.

Conclusions: Both tobacco and alcohol induce a similar increase of intracellular calcium levels in pancreatic acinar cell, but this is associated with an increased amylase secretion only with tobacco. Alcohol and tobacco have a different effect in pancreatic acinar cells and should play a different role in the pathophysiology of CP.

P27.

acini signiﬁcantly and dose-dependently downregulated GHS-R1a, but failed to affect the signal for ghrelin. Pretreatment of the rats with ghrelin prevented from caerulein-induced downregulation of GHS-R1a.

Conclusions: Caerulein is able to modify the GHS-R1a subtype in the pancreatic acini. This effect could be prevented by pretreatment with ghrelin and perhaps might be implicated in the mechanism of pancreatic damage induced by caerulein overstimulation.

P28.

Changes of hemorheological laboratory parameters and course of medication in experimental acute pancreatitis R. Kotan , N. Nemeth , F. Kiss , J. Posan , K. Miszti-Blasius , L. Toth 4, I. Furka , I. Miko, P. Sapy, Zs. Szentkereszty1.



 Institute of Surgery, Medical and Health Science Center, University of Debrecen, Hungary  Department of Operative Techniques and Surgical Research, Medical and Health Science Center, University of Debrecen, Hungary  Department of Clinical Biochemistry and Molecular Pathology, Medical and Health Science Center, University of Debrecen, Hungary 4 Department of Pathology, Medical and Health Science Center, University of Debrecen, Hungary Introduction: The role of microcirculatory disturbance in severe acute pancreatitis is generally accepted.



Objectives: Effectiveness of Pentoxifyllin, Enoxaparine and Flunixin was analysed in experimental cerulein pancreatitis.

Methods: Fifty female Sprague-Dawley rats divided into 5 groups . In general anaesthesia 2 hours after acute pancreatitis was induced $(\mathbf{n}{=}10)$  tissue microcirculationwas examined, cupping, was performed in all group. Control group (C): no pancreatitis. Acute pancreatitis group (AP):  $10\,\upmu\mathrm{g/kg}$  ceruleinwas injected s.c.  $\mathrm{AP+}$ Flunixin group  $(\mathsf{A P}\mathrm{+}\mathsf{F l})$ :  $2{,}5\,\mathrm{mg/kg}$ Flunixinwas also administered. AP+Pentoxifyllin group (AP+Pe):  $50\,\mathrm{mg/kg}$ Pentoxifyllin i.p. was also administered. AP+Clexan group  $(\mathrm{AP+E})$ :  $2\,\mathrm{mg/kg}$ Enoxaparine s.c. was also administered. Tissue blood ﬂow unit (BFU), rectal temperature, blood acidity and hemorheological parameters were measured.

Results: The rectal temperature was signiﬁcantly elevated in AP, it was moderate in AP+Fl and  $\mathrm{AP+E}$ and high in AP+Pe. The decrease of BFU was lower in  $\mathrm{AP+Pe}$ and  $\mathsf{A P+F l}.$  The blood pH level decrease was moderate in all treated groups. The elevation of hematocrit level was moderate in  $\scriptstyle{\mathrm{AP}}+\mathrm{Fl}$  The maximal elongation index values decrease in  $\scriptstyle{\mathrm{AP}}+{\mathrm{Fl}}$ was not noticed. The osmoscan conﬁrmed the best result also in AP+Fl. The  $\mathsf{M}$  and M1 indices were elevated in all AP, it was moderate in  $\scriptstyle\mathrm{AP+Fl}$ and signiﬁcant in  $\mathrm{AP+Pe}$ . The changes of the RBC aggregation parameters were resemble.

Conclusion: The measured drugs had good effects on microcirculation, pH levels and microrheologic parameters in cerulein induced acute pancre- atitis in rats.

P29.
Intracellular calcium levels and enzyme secretion in response to tobacco vs ethanol in pancreatic acinar cells M. Luaces-Regueira , M. Castiñeira-Alvariño , M. Castro-Manzanares 2, A. Carrascal-Miniño , M. Campos-Toimil, J.E. Domínguez-Muñoz1.



 Foundation for Research in Digestive Diseases, Department of Gastroenterology, University Hospital of Santiago de Compostela, Spain  Farmacology Department, University of Santiago de Compostela, Spain Alterations in intracellular calcium levels and enzyme secretion in acinar cells are involved in the pathophysiology of chronic pancreatitis (CP). It has been previously shown that alcohol induces an increase of intracellular calcium, but its effect in enzyme secretion is controversial. The effect of tobacco in this context is unknown.



Hypothesis: Tobacco, similarly to alcohol, promotes the development of CP by altering intracellular calcium levels and enzyme secretion in acinar cells.

Aim: To evaluate the effect of tobacco compared with alcohol in intracellular calcium levels and amylase secretion in pancreatic acinar cells.

Methods: Experimental in-vitro study on primary acinar cell culture. Acinar cells were isolated by enzymatic and mechanic degradation from Swiss mice pancreas. Cells were stimulated by increasing concentrations of alcohol ( $.10{-}100\ \mathrm{mM})$  and tobacco  $(0.1{-}0.5~\mathrm{mg/ml})$ , and with CCK as posi- tive control. Intracellular calcium levels were measured by ﬂuorescence andamylasesecretionbyusing $\mathfrak{p}$ -nitrophenyl-maltohexaosideas substrate. Data were analysed by ANOVA.

Results: Tobacco induced a dose-dependent increase of intracellular calcium levels from  $11.38{\pm}4.89\%\left(0.1\;\mathrm{mg/ml}\right)$  to  $56.26{\pm}13.14\%\left(0.5\;\mathrm{mg/ml}\right)$ .  ( $10\,\mathrm{\mM})$  to $[14.40\pm2.06\%$   $59.8{\pm}2.57\%\left(75\mathrm{mM}\right)]$  with tobacco (up to  $21\%$  at  $0.4\;\mathrm{mg/ml})$ , but not with alcohol.

Conclusions: Both tobacco and alcohol induce a similar increase of intracellular calcium levels in pancreatic acinar cell, but this is associated with an increased amylase secretion only with tobacco. Alcohol and tobacco have a different effect in pancreatic acinar cells and should play a different role in the pathophysiology of CP.

P27.

acini signiﬁcantly and dose-dependently downregulated GHS-R1a, but failed to affect the signal for ghrelin. Pretreatment of the rats with ghrelin prevented from caerulein-induced downregulation of GHS-R1a.

Conclusions: Caerulein is able to modify the GHS-R1a subtype in the pancreatic acini. This effect could be prevented by pretreatment with ghrelin and perhaps might be implicated in the mechanism of pancreatic damage induced by caerulein overstimulation.

P28.

Changes of hemorheological laboratory parameters and course of medication in experimental acute pancreatitis R. Kotan , N. Nemeth , F. Kiss , J. Posan , K. Miszti-Blasius , L. Toth 4, I. Furka , I. Miko, P. Sapy, Zs. Szentkereszty1.



 Institute of Surgery, Medical and Health Science Center, University of Debrecen, Hungary  Department of Operative Techniques and Surgical Research, Medical and Health Science Center, University of Debrecen, Hungary  Department of Clinical Biochemistry and Molecular Pathology, Medical and Health Science Center, University of Debrecen, Hungary 4 Department of Pathology, Medical and Health Science Center, University of Debrecen, Hungary Introduction: The role of microcirculatory disturbance in severe acute pancreatitis is generally accepted.



Objectives: Effectiveness of Pentoxifyllin, Enoxaparine and Flunixin was analysed in experimental cerulein pancreatitis.

Methods: Fifty female Sprague-Dawley rats divided into 5 groups . In general anaesthesia 2 hours after acute pancreatitis was induced $(\mathbf{n}{=}10)$  tissue microcirculationwas examined, cupping, was performed in all group. Control group (C): no pancreatitis. Acute pancreatitis group (AP):  $10\,\upmu\mathrm{g/kg}$  ceruleinwas injected s.c.  $\mathrm{AP+}$ Flunixin group  $(\mathsf{A P}\mathrm{+}\mathsf{F l})$ :  $2{,}5\,\mathrm{mg/kg}$ Flunixinwas also administered. AP+Pentoxifyllin group (AP+Pe):  $50\,\mathrm{mg/kg}$ Pentoxifyllin i.p. was also administered. AP+Clexan group  $(\mathrm{AP+E})$ :  $2\,\mathrm{mg/kg}$ Enoxaparine s.c. was also administered. Tissue blood ﬂow unit (BFU), rectal temperature, blood acidity and hemorheological parameters were measured.

Results: The rectal temperature was signiﬁcantly elevated in AP, it was moderate in AP+Fl and  $\mathrm{AP+E}$ and high in AP+Pe. The decrease of BFU was lower in  $\mathrm{AP+Pe}$ and  $\mathsf{A P+F l}.$  The blood pH level decrease was moderate in all treated groups. The elevation of hematocrit level was moderate in  $\scriptstyle{\mathrm{AP}}+\mathrm{Fl}$  The maximal elongation index values decrease in  $\scriptstyle{\mathrm{AP}}+{\mathrm{Fl}}$ was not noticed. The osmoscan conﬁrmed the best result also in AP+Fl. The  $\mathsf{M}$  and M1 indices were elevated in all AP, it was moderate in  $\scriptstyle\mathrm{AP+Fl}$ and signiﬁcant in  $\mathrm{AP+Pe}$ . The changes of the RBC aggregation parameters were resemble.

Conclusion: The measured drugs had good effects on microcirculation, pH levels and microrheologic parameters in cerulein induced acute pancre- atitis in rats.

P29.
Intracellular calcium levels and enzyme secretion in response to tobacco vs ethanol in pancreatic acinar cells M. Luaces-Regueira , M. Castiñeira-Alvariño , M. Castro-Manzanares 2, A. Carrascal-Miniño , M. Campos-Toimil, J.E. Domínguez-Muñoz1.



 Foundation for Research in Digestive Diseases, Department of Gastroenterology, University Hospital of Santiago de Compostela, Spain  Farmacology Department, University of Santiago de Compostela, Spain Alterations in intracellular calcium levels and enzyme secretion in acinar cells are involved in the pathophysiology of chronic pancreatitis (CP). It has been previously shown that alcohol induces an increase of intracellular calcium, but its effect in enzyme secretion is controversial. The effect of tobacco in this context is unknown.



Hypothesis: Tobacco, similarly to alcohol, promotes the development of CP by altering intracellular calcium levels and enzyme secretion in acinar cells.

Aim: To evaluate the effect of tobacco compared with alcohol in intracellular calcium levels and amylase secretion in pancreatic acinar cells.

Methods: Experimental in-vitro study on primary acinar cell culture. Acinar cells were isolated by enzymatic and mechanic degradation from Swiss mice pancreas. Cells were stimulated by increasing concentrations of alcohol ( $.10{-}100\ \mathrm{mM})$  and tobacco  $(0.1{-}0.5~\mathrm{mg/ml})$ , and with CCK as posi- tive control. Intracellular calcium levels were measured by ﬂuorescence andamylasesecretionbyusing $\mathfrak{p}$ -nitrophenyl-maltohexaosideas substrate. Data were analysed by ANOVA.

Results: Tobacco induced a dose-dependent increase of intracellular calcium levels from  $11.38{\pm}4.89\%\left(0.1\;\mathrm{mg/ml}\right)$  to  $56.26{\pm}13.14\%\left(0.5\;\mathrm{mg/ml}\right)$ .  ( $10\,\mathrm{\mM})$  to $[14.40\pm2.06\%$   $59.8{\pm}2.57\%\left(75\mathrm{mM}\right)]$  with tobacco (up to  $21\%$  at  $0.4\;\mathrm{mg/ml})$ , but not with alcohol.

Conclusions: Both tobacco and alcohol induce a similar increase of intracellular calcium levels in pancreatic acinar cell, but this is associated with an increased amylase secretion only with tobacco. Alcohol and tobacco have a different effect in pancreatic acinar cells and should play a different role in the pathophysiology of CP.

P27.

acini signiﬁcantly and dose-dependently downregulated GHS-R1a, but failed to affect the signal for ghrelin. Pretreatment of the rats with ghrelin prevented from caerulein-induced downregulation of GHS-R1a.

Conclusions: Caerulein is able to modify the GHS-R1a subtype in the pancreatic acini. This effect could be prevented by pretreatment with ghrelin and perhaps might be implicated in the mechanism of pancreatic damage induced by caerulein overstimulation.

P28.

Changes of hemorheological laboratory parameters and course of medication in experimental acute pancreatitis R. Kotan , N. Nemeth , F. Kiss , J. Posan , K. Miszti-Blasius , L. Toth 4, I. Furka , I. Miko, P. Sapy, Zs. Szentkereszty1.



 Institute of Surgery, Medical and Health Science Center, University of Debrecen, Hungary  Department of Operative Techniques and Surgical Research, Medical and Health Science Center, University of Debrecen, Hungary  Department of Clinical Biochemistry and Molecular Pathology, Medical and Health Science Center, University of Debrecen, Hungary 4 Department of Pathology, Medical and Health Science Center, University of Debrecen, Hungary Introduction: The role of microcirculatory disturbance in severe acute pancreatitis is generally accepted.



Objectives: Effectiveness of Pentoxifyllin, Enoxaparine and Flunixin was analysed in experimental cerulein pancreatitis.

Methods: Fifty female Sprague-Dawley rats divided into 5 groups . In general anaesthesia 2 hours after acute pancreatitis was induced $(\mathbf{n}{=}10)$  tissue microcirculationwas examined, cupping, was performed in all group. Control group (C): no pancreatitis. Acute pancreatitis group (AP):  $10\,\upmu\mathrm{g/kg}$  ceruleinwas injected s.c.  $\mathrm{AP+}$ Flunixin group  $(\mathsf{A P}\mathrm{+}\mathsf{F l})$ :  $2{,}5\,\mathrm{mg/kg}$ Flunixinwas also administered. AP+Pentoxifyllin group (AP+Pe):  $50\,\mathrm{mg/kg}$ Pentoxifyllin i.p. was also administered. AP+Clexan group  $(\mathrm{AP+E})$ :  $2\,\mathrm{mg/kg}$ Enoxaparine s.c. was also administered. Tissue blood ﬂow unit (BFU), rectal temperature, blood acidity and hemorheological parameters were measured.

Results: The rectal temperature was signiﬁcantly elevated in AP, it was moderate in AP+Fl and  $\mathrm{AP+E}$ and high in AP+Pe. The decrease of BFU was lower in  $\mathrm{AP+Pe}$ and  $\mathsf{A P+F l}.$  The blood pH level decrease was moderate in all treated groups. The elevation of hematocrit level was moderate in  $\scriptstyle{\mathrm{AP}}+\mathrm{Fl}$  The maximal elongation index values decrease in  $\scriptstyle{\mathrm{AP}}+{\mathrm{Fl}}$ was not noticed. The osmoscan conﬁrmed the best result also in AP+Fl. The  $\mathsf{M}$  and M1 indices were elevated in all AP, it was moderate in  $\scriptstyle\mathrm{AP+Fl}$ and signiﬁcant in  $\mathrm{AP+Pe}$ . The changes of the RBC aggregation parameters were resemble.

Conclusion: The measured drugs had good effects on microcirculation, pH levels and microrheologic parameters in cerulein induced acute pancre- atitis in rats.

P29.
Intracellular calcium levels and enzyme secretion in response to tobacco vs ethanol in pancreatic acinar cells M. Luaces-Regueira , M. Castiñeira-Alvariño , M. Castro-Manzanares 2, A. Carrascal-Miniño , M. Campos-Toimil, J.E. Domínguez-Muñoz1.



 Foundation for Research in Digestive Diseases, Department of Gastroenterology, University Hospital of Santiago de Compostela, Spain  Farmacology Department, University of Santiago de Compostela, Spain Alterations in intracellular calcium levels and enzyme secretion in acinar cells are involved in the pathophysiology of chronic pancreatitis (CP). It has been previously shown that alcohol induces an increase of intracellular calcium, but its effect in enzyme secretion is controversial. The effect of tobacco in this context is unknown.



Hypothesis: Tobacco, similarly to alcohol, promotes the development of CP by altering intracellular calcium levels and enzyme secretion in acinar cells.

Aim: To evaluate the effect of tobacco compared with alcohol in intracellular calcium levels and amylase secretion in pancreatic acinar cells.

Methods: Experimental in-vitro study on primary acinar cell culture. Acinar cells were isolated by enzymatic and mechanic degradation from Swiss mice pancreas. Cells were stimulated by increasing concentrations of alcohol ( $.10{-}100\ \mathrm{mM})$  and tobacco  $(0.1{-}0.5~\mathrm{mg/ml})$ , and with CCK as posi- tive control. Intracellular calcium levels were measured by ﬂuorescence andamylasesecretionbyusing $\mathfrak{p}$ -nitrophenyl-maltohexaosideas substrate. Data were analysed by ANOVA.

Results: Tobacco induced a dose-dependent increase of intracellular calcium levels from  $11.38{\pm}4.89\%\left(0.1\;\mathrm{mg/ml}\right)$  to  $56.26{\pm}13.14\%\left(0.5\;\mathrm{mg/ml}\right)$ .  ( $10\,\mathrm{\mM})$  to $[14.40\pm2.06\%$   $59.8{\pm}2.57\%\left(75\mathrm{mM}\right)]$  with tobacco (up to  $21\%$  at  $0.4\;\mathrm{mg/ml})$ , but not with alcohol.

Conclusions: Both tobacco and alcohol induce a similar increase of intracellular calcium levels in pancreatic acinar cell, but this is associated with an increased amylase secretion only with tobacco. Alcohol and tobacco have a different effect in pancreatic acinar cells and should play a different role in the pathophysiology of CP.

P27.

acini signiﬁcantly and dose-dependently downregulated GHS-R1a, but failed to affect the signal for ghrelin. Pretreatment of the rats with ghrelin prevented from caerulein-induced downregulation of GHS-R1a.

Conclusions: Caerulein is able to modify the GHS-R1a subtype in the pancreatic acini. This effect could be prevented by pretreatment with ghrelin and perhaps might be implicated in the mechanism of pancreatic damage induced by caerulein overstimulation.

P28.

Changes of hemorheological laboratory parameters and course of medication in experimental acute pancreatitis R. Kotan , N. Nemeth , F. Kiss , J. Posan , K. Miszti-Blasius , L. Toth 4, I. Furka , I. Miko, P. Sapy, Zs. Szentkereszty1.



 Institute of Surgery, Medical and Health Science Center, University of Debrecen, Hungary  Department of Operative Techniques and Surgical Research, Medical and Health Science Center, University of Debrecen, Hungary  Department of Clinical Biochemistry and Molecular Pathology, Medical and Health Science Center, University of Debrecen, Hungary 4 Department of Pathology, Medical and Health Science Center, University of Debrecen, Hungary Introduction: The role of microcirculatory disturbance in severe acute pancreatitis is generally accepted.



Objectives: Effectiveness of Pentoxifyllin, Enoxaparine and Flunixin was analysed in experimental cerulein pancreatitis.

Methods: Fifty female Sprague-Dawley rats divided into 5 groups . In general anaesthesia 2 hours after acute pancreatitis was induced $(\mathbf{n}{=}10)$  tissue microcirculationwas examined, cupping, was performed in all group. Control group (C): no pancreatitis. Acute pancreatitis group (AP):  $10\,\upmu\mathrm{g/kg}$  ceruleinwas injected s.c.  $\mathrm{AP+}$ Flunixin group  $(\mathsf{A P}\mathrm{+}\mathsf{F l})$ :  $2{,}5\,\mathrm{mg/kg}$ Flunixinwas also administered. AP+Pentoxifyllin group (AP+Pe):  $50\,\mathrm{mg/kg}$ Pentoxifyllin i.p. was also administered. AP+Clexan group  $(\mathrm{AP+E})$ :  $2\,\mathrm{mg/kg}$ Enoxaparine s.c. was also administered. Tissue blood ﬂow unit (BFU), rectal temperature, blood acidity and hemorheological parameters were measured.

Results: The rectal temperature was signiﬁcantly elevated in AP, it was moderate in AP+Fl and  $\mathrm{AP+E}$ and high in AP+Pe. The decrease of BFU was lower in  $\mathrm{AP+Pe}$ and  $\mathsf{A P+F l}.$  The blood pH level decrease was moderate in all treated groups. The elevation of hematocrit level was moderate in  $\scriptstyle{\mathrm{AP}}+\mathrm{Fl}$  The maximal elongation index values decrease in  $\scriptstyle{\mathrm{AP}}+{\mathrm{Fl}}$ was not noticed. The osmoscan conﬁrmed the best result also in AP+Fl. The  $\mathsf{M}$  and M1 indices were elevated in all AP, it was moderate in  $\scriptstyle\mathrm{AP+Fl}$ and signiﬁcant in  $\mathrm{AP+Pe}$ . The changes of the RBC aggregation parameters were resemble.

Conclusion: The measured drugs had good effects on microcirculation, pH levels and microrheologic parameters in cerulein induced acute pancre- atitis in rats.

P29.
AUTHORS:  Patrick A. Woudt Rob P. Fender Richard P. Armstrong

 



Claude Carignan1

### AFFILIATIONS:

Astrophysics, Cosmology and  Gravity Centre, Department of  Astronomy, University of Cape  Town, Cape Town, South Africa 2Physics and Astronomy,  University of Southampton, High  Field, United Kingdom SKA South Africa, Pinelands,  South Africa CORRESPONDENCE TO:  Patrick Woudt EMAIL:  Patrick.Woudt@uct.ac.za POSTAL ADDRESS:  Department of Astronomy,  University of Cape Town, Private  Bag X3, Rondebosch 7701,  South Africa HOW TO CITE: Woudt PA, Fender RP, Armstrong  RP, Carignan C. Early science  with the Karoo Array Telescope  test array KAT-7. S Afr J Sci. 

     

    

   

  

 



2013;109(7/8), Art. #a0030, 2  pages. http://dx.doi.org/10.1590/ sajs.2013/a0030

# Early science with the Karoo Array   Telescope test array KAT-7

One year after the site selection decision of the Square Kilometre Array (SKA), the first significant scientific results  from South Africa’s Karoo Array Telescope test array KAT-7 have appeared in the astronomical literature. This  marks one of the many major milestones on the path to the SKA in Africa and highlights the commencement of  cutting-edge science observations with South Africa’s new radio interferometer.

The SKA project is the largest basic science project ever undertaken in South Africa1 and has been developing  in a number of closely linked phases: (1) the construction (Carnavon) and remote operation (Cape Town) of the  technology demonstrator KAT-7, (2) the highly essential human capacity development programme in South Africa  and associated SKA Africa partner countries (Youth in Science and Engineering), (3) the construction (Carnavon)  and remote operation (Cape Town) of the SKA-precursor array MeerKAT and (4) the integration of the MeerKAT  array into the first phase of the international SKA project. The potential spin-offs from this project for other areas of  science and industry are tremendous, not least through the computational requirements for large-scale processing  of ‘big data’ in real time. 

The construction of the technology demonstrator KAT-7 – a radio interferometer consisting of seven fully steerable  12-m dishes built on-site in Carnavon – was completed in 2011. Following engineering commissioning of the  new array, science commissioning, focusing on testing a wide range of operational modes required for MeerKAT  science, started in 2012. Designed as a technology and system demonstrator for MeerKAT, KAT-7 was completed  on time and within budget by a team of young engineers, who, in the process of building KAT-7, have gained much  of the essential experience to construct MeerKAT to full SKA specifications. KAT-7 now moves beyond its initial  role of an engineering test facility to full science operations involving the growing community of radio astronomers,  physicists and engineers in South Africa.

The human capacity development programme of the SKA SA project office, initiated in 2005, is an integral part of the  SKA in Africa. By 2012 this programme delivered scholarships to over 400 students and fellowships to 22 postdoctoral  researchers in astrophysics, physics and engineering. Given the success of this programme and the long-term  investment by the Department of Science and Technology in the training of skilled SKA scientists and engineers in  South Africa, it is not surprising that SKA SA scholars have fulfilled key roles in the recent KAT-7 publications. 

In 2009, the SKA SA project released an international call5 for large legacy-style projects on MeerKAT during the  first 5 years of its scientific operation. A total of 21 proposals were received from the international community and  after peer-review 10 projects were selected − hereafter referred to as the MeerKAT Large Survey Projects (LSP) –  which cover a range of science topics at the very heart of the SKA science case. The MeerKAT LSPs represent large  international teams of astronomers, physicists and engineers combining a broad spectrum of expertise required for  SKA science. Four of the MeerKAT LSPs are co-led by astronomers in South Africa. The South African scientists  involved in the two recent papers are all part of teams preparing for the implementation of the MeerKAT LSPs and  are closely involved in the science commissioning of KAT-7. 

In the first of the two recent papers, the KAT-7 array was used in combination with the 26-m Hartebeeshoek  Radio Astronomy Observatory (HartRAO) radio telescope near Johannesburg to study the energetic outflow of  matter from Circinus X-1, an accreting neutron star in a compact binary system with a highly eccentric orbit.  A daily monitoring campaign over a period in excess of 1 month (13 December 2011 – 16 January 2012) showed  strong repeated flares from this binary system (Figure 1), modelled as relativistic jets originating from the neutron  star every time the companion star reached periastron. After decades of low-level activity, this binary system  has returned to a state of bright flares, presumably as a result of changes in the accretion rate of matter onto the  neutron star. It is an ideal object to study with the KAT-7 array over extended periods of time; the brightness of  the flares is a good match to the sensitivity of the KAT-7 array and the object itself is one of the best laboratories  to study relativistic jet astrophysics on an accessible timescale (hours to days to months). The study of transient  phenomena in the radio sky is one of the niche areas of research for KAT-7, and the scientific focus of two of the  MeerKAT LSPs.

The second paper recently published reported  $120~\mathsf{h}$ of KAT-7 observations of the nearby galaxy NGC 3109  (Figure 2). Mapping the distribution of its neutral hydrogen gas content through a spectral line transition (the spin  flip of the electron in atomic hydrogen), astronomers were able to model the dark matter distribution in this galaxy.  The amount and distribution of dark matter is inferred from the kinematics of the neutral hydrogen gas traced from  the centre of the galaxy out to large radial distances; in the case of NGC 3109, neutral hydrogen was measured  with KAT-7 out to much larger radial distances than observed before. Because of the compact design of the KAT-7  array, it is more sensitive to faint extended emission compared with other radio interferometers. Combined with  a large field of view and good receivers, mapping neutral hydrogen in nearby galaxies is another niche area for  KAT-7. The KAT-7 observations of NGC 3109 put stronger constraints on an alternative explanation for dark matter,  namely Modified Newtonian Dynamics (MOND). MOND cannot represent the observed rotational velocities of the  gas in NGC 3109. 

KAT-7 is now at a stage where it is able to respond regularly to requests from the international community for  observations of specific objects. Flexible scheduling in combination with state-of-the-art receivers will make KAT-7  a very attractive radio telescope until MeerKAT comes online. The MeerKAT LSPs will continue to look to KAT-7 to  test new and novel data analysis techniques ultimately designed for MeerKAT. 

Copyright of South African Journal of Science is the property of Academy of Science of South Africa and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
AUTHORS:  Patrick A. Woudt Rob P. Fender Richard P. Armstrong

 



Claude Carignan1

### AFFILIATIONS:

Astrophysics, Cosmology and  Gravity Centre, Department of  Astronomy, University of Cape  Town, Cape Town, South Africa 2Physics and Astronomy,  University of Southampton, High  Field, United Kingdom SKA South Africa, Pinelands,  South Africa CORRESPONDENCE TO:  Patrick Woudt EMAIL:  Patrick.Woudt@uct.ac.za POSTAL ADDRESS:  Department of Astronomy,  University of Cape Town, Private  Bag X3, Rondebosch 7701,  South Africa HOW TO CITE: Woudt PA, Fender RP, Armstrong  RP, Carignan C. Early science  with the Karoo Array Telescope  test array KAT-7. S Afr J Sci. 

     

    

   

  

 



2013;109(7/8), Art. #a0030, 2  pages. http://dx.doi.org/10.1590/ sajs.2013/a0030

# Early science with the Karoo Array   Telescope test array KAT-7

One year after the site selection decision of the Square Kilometre Array (SKA), the first significant scientific results  from South Africa’s Karoo Array Telescope test array KAT-7 have appeared in the astronomical literature. This  marks one of the many major milestones on the path to the SKA in Africa and highlights the commencement of  cutting-edge science observations with South Africa’s new radio interferometer.

The SKA project is the largest basic science project ever undertaken in South Africa1 and has been developing  in a number of closely linked phases: (1) the construction (Carnavon) and remote operation (Cape Town) of the  technology demonstrator KAT-7, (2) the highly essential human capacity development programme in South Africa  and associated SKA Africa partner countries (Youth in Science and Engineering), (3) the construction (Carnavon)  and remote operation (Cape Town) of the SKA-precursor array MeerKAT and (4) the integration of the MeerKAT  array into the first phase of the international SKA project. The potential spin-offs from this project for other areas of  science and industry are tremendous, not least through the computational requirements for large-scale processing  of ‘big data’ in real time. 

The construction of the technology demonstrator KAT-7 – a radio interferometer consisting of seven fully steerable  12-m dishes built on-site in Carnavon – was completed in 2011. Following engineering commissioning of the  new array, science commissioning, focusing on testing a wide range of operational modes required for MeerKAT  science, started in 2012. Designed as a technology and system demonstrator for MeerKAT, KAT-7 was completed  on time and within budget by a team of young engineers, who, in the process of building KAT-7, have gained much  of the essential experience to construct MeerKAT to full SKA specifications. KAT-7 now moves beyond its initial  role of an engineering test facility to full science operations involving the growing community of radio astronomers,  physicists and engineers in South Africa.

The human capacity development programme of the SKA SA project office, initiated in 2005, is an integral part of the  SKA in Africa. By 2012 this programme delivered scholarships to over 400 students and fellowships to 22 postdoctoral  researchers in astrophysics, physics and engineering. Given the success of this programme and the long-term  investment by the Department of Science and Technology in the training of skilled SKA scientists and engineers in  South Africa, it is not surprising that SKA SA scholars have fulfilled key roles in the recent KAT-7 publications. 

In 2009, the SKA SA project released an international call5 for large legacy-style projects on MeerKAT during the  first 5 years of its scientific operation. A total of 21 proposals were received from the international community and  after peer-review 10 projects were selected − hereafter referred to as the MeerKAT Large Survey Projects (LSP) –  which cover a range of science topics at the very heart of the SKA science case. The MeerKAT LSPs represent large  international teams of astronomers, physicists and engineers combining a broad spectrum of expertise required for  SKA science. Four of the MeerKAT LSPs are co-led by astronomers in South Africa. The South African scientists  involved in the two recent papers are all part of teams preparing for the implementation of the MeerKAT LSPs and  are closely involved in the science commissioning of KAT-7. 

In the first of the two recent papers, the KAT-7 array was used in combination with the 26-m Hartebeeshoek  Radio Astronomy Observatory (HartRAO) radio telescope near Johannesburg to study the energetic outflow of  matter from Circinus X-1, an accreting neutron star in a compact binary system with a highly eccentric orbit.  A daily monitoring campaign over a period in excess of 1 month (13 December 2011 – 16 January 2012) showed  strong repeated flares from this binary system (Figure 1), modelled as relativistic jets originating from the neutron  star every time the companion star reached periastron. After decades of low-level activity, this binary system  has returned to a state of bright flares, presumably as a result of changes in the accretion rate of matter onto the  neutron star. It is an ideal object to study with the KAT-7 array over extended periods of time; the brightness of  the flares is a good match to the sensitivity of the KAT-7 array and the object itself is one of the best laboratories  to study relativistic jet astrophysics on an accessible timescale (hours to days to months). The study of transient  phenomena in the radio sky is one of the niche areas of research for KAT-7, and the scientific focus of two of the  MeerKAT LSPs.

The second paper recently published reported  $120~\mathsf{h}$ of KAT-7 observations of the nearby galaxy NGC 3109  (Figure 2). Mapping the distribution of its neutral hydrogen gas content through a spectral line transition (the spin  flip of the electron in atomic hydrogen), astronomers were able to model the dark matter distribution in this galaxy.  The amount and distribution of dark matter is inferred from the kinematics of the neutral hydrogen gas traced from  the centre of the galaxy out to large radial distances; in the case of NGC 3109, neutral hydrogen was measured  with KAT-7 out to much larger radial distances than observed before. Because of the compact design of the KAT-7  array, it is more sensitive to faint extended emission compared with other radio interferometers. Combined with  a large field of view and good receivers, mapping neutral hydrogen in nearby galaxies is another niche area for  KAT-7. The KAT-7 observations of NGC 3109 put stronger constraints on an alternative explanation for dark matter,  namely Modified Newtonian Dynamics (MOND). MOND cannot represent the observed rotational velocities of the  gas in NGC 3109. 

KAT-7 is now at a stage where it is able to respond regularly to requests from the international community for  observations of specific objects. Flexible scheduling in combination with state-of-the-art receivers will make KAT-7  a very attractive radio telescope until MeerKAT comes online. The MeerKAT LSPs will continue to look to KAT-7 to  test new and novel data analysis techniques ultimately designed for MeerKAT. 

Copyright of South African Journal of Science is the property of Academy of Science of South Africa and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
# The toxic effects of nonylphenol on the reproductive system of male rats

Xiao D. Han,∗, Zhi G. Tu, Yi Gong, Su N. Shen, Xu Y. Wang, Li N. Kang, Ya Y. Hou, Jian X. Chen

 Immunology and Reproductive Biology Laboratory, Medical School, Nanjing University, Nanjing, Jiangsu 210093, PR China  Department of Biology, School of Life Science, Nanjing University, Nanjing, Jiangsu 210093, PR China Received 20 November 2003; received in revised form 16 April 2004; accepted 25 June 2004 Available online 19 August 2004



### Abstract

Male Sprague–Dawley (SD) rats were exposed to $^{4-t}$ -nonylphenol (NP) by gavage at dosages of 0, 125 and  $250\,\mathrm{mg/kg/day}$  for 50 days. Organ weights of liver, kidney, testis and epididymis were measured. Sperm number in the head of epididymis was counted. Several hormones including testosterone, luteinizing hormone and follicle stimulating hormone were measured. Testicular sections were observed by light and electron microscopy. Terminal dideoxynucleotidyl transferase dUTP nick end labeling assay was performed to probe the apoptotic cells in seminiferous tubules. When rats were treated with nonylphenol at  $250\,\mathrm{mg/kg/day},$ , the absolute and relative weight of epididymis decreased dramatically, while the relative weights of kidney and liver increased by 14 and $22\%$ , respectively. In addition, the sperm density of the head of epididymis and the testosterone level descended at  $250\,\mathrm{mg/kg/day}$ . The levels of luteinizing hormone and follicle stimulating hormone increased in both nonylphenol treated groups. Pathological changes were detected by microscopy and the transferase dUTP nick end labeling (TUNEL) assay showed that the number of apoptotic cells in testes increased with nonylphenol in a dose-dependent manner.

 $\copyright2004$  Elsevier Inc. All rights reserved.

Keywords: Nonylphonol; Apoptosis; Reproductive system; Testosterone; Luteinizing hormone; Follicle stimulating hormone; TUNEL

## 1. Introduction

Nonylphenol polyethoxylates (NPEs) are non-ionic sur- factants that are widely used as components of detergents, paints, herbicides, insecticides and many other synthetic products. With their wide use, a large amount of NPEs are charged into water. Nonylphenol (NP), a primary ﬁnal biodegradation product of NPEs [1], has been documented to appear in aquatic environment, particularly in sediment. Its content may reach  $3000\;\mathrm{ppb}$  in rivers and lakes [2]. NP showed a great capacity of bioaccumulation in aquatic an- imals, and investigators found that the NP concentration in the muscle of Anas boscas (living in contaminated aquatic environment) reached  $1.2\,\mathrm{mg/kg}$  [3].

NP has been reported to be weakly estrogenic from in vitro and in vivo studies. In Chapin’s study [4], 3.5 generations of rats were exposed to NP in the diet at 200, 650 and $2000\,\mathrm{ppm}$ , which corresponded to dietary intakes of 9–35, 30–100, and  $100{-}350\,\mathrm{mg/kg/day}$ . Vaginal opening was accelerated by 2 days (at  $650\,\mathrm{ppm})$  and 6 days (at $2000\,\mathrm{ppm})$ ). Uterine weights at postnatal day 21 were increased in F1 females treated with 650 and 2000 ppm NP. In F2 rats, epididymal sperm density was reduced by 8 and  $13\%$  at 650 and 2000 ppm, respec- tively. Collectively, these results suggest that NP has weak estrogenic activity and may induce abnormities in female and male reproductive systems. In Hossaini’s study [5], female pregnant rats were dosed with NP at 3, 15, or  $75\,\mathrm{mg/kg/c}$ day from gestation days 11 to 18. Male offsprings were sacri- ﬁced at postnatal days 11, 21, and 110, and their reproduc- tive parameters were evaluated. The absolute weight of the right epididymis signiﬁcantly decreased in the animals whose mother was exposed to 75 or 15 mg/kg/day NP. In another study,  $100\,\mathrm{mg/kg/day}\,\mathrm{N}$ P reduced the daily sperm production of male rats, and  $300\,\mathrm{mg/kg/day}\ \mathrm{N}$ P induced the atrophy of testis and epididymis [6]. Until now, systemic studies on the male reproductive toxicity of NP have been scarce. The aim of this present study is to further investigate the mechanisms of the reproductive toxicity of NP in male rats.



## 2. Materials and methods

## 2.1. Instruments and reagents

Instruments: Roche Elecsys System (2010), Electron microscope (JEM-1200EX),  $\upgamma$ -counter (SN-695). Sources of reagents were as follows: corn oil (Fortune, Shang- hai, PR China); technical nonylphenol (FGJ 01, purity:  $98\%$ , Tokyo Kasei Kogyo Co. Ltd.); pentobarbital (Sigma, USA); testosterone reagent kit (Roche Diagnostics GmbH, D-68298 Mannheim, Germany); human luteinizing hor- mone radioimmunoassay kit and follicle stimulating hor- mone radioimmunoassay kit (Jiuding Co. Ltd., PR China); apoptosis detection kit (R&D systems, Minneapolis, MN, USA).

## 2.2. Animal treatments

Thirty male Sprague–Dawley rats, aging 28–30 days, were purchased from Nanjing Medical University and housed in groups at  $24\,^{\circ}\mathrm{C}$  in a humidity-controlled room with 12 h light/dark photoperiod. Standard rat chow was ob- tained from Nanjing Medical University (Nanjing Medical University, Nanjing, P.R. China) and available throughout the experiment. Rats were acclimated for 1 week prior to treatments.

Male rats were divided into three groups: control, NP1 and NP2. NP was dissolved in corn oil and given to rats by gavage at  $0\,\mathrm{mg/kg/day}$  (control),  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2). Gavage volume was adjusted accord- ing to the weight of each rat. During the experiment, one rat in the NP2 group manifested a marked reduction in body weight gain and was excluded from the formal experiment. After 50 days treatment, the rats were weighed and anaesthetized by injecting pentobarbital. So far, no report indicates that pen- tobarbital can affect the levels of LH, FSH and testosterone. To obtain pure serum, blood samples were collected via the carotid artery. We inserted a ﬁne tube into the carotid to draw blood into a test tube. When the content of extracted blood reached $3.5–4.5\%$  body weight, this operation was regarded as successful; else it was designated a failure. Because the residual blood in the body will affect organ weight, the cases called failure were excluded from the control, NP1 and NP2 groups as follows: 2, 2 and 1, cases, respectively. At the end of this study, each group had eight rats that can offer available data. After this step, liver, kidney, testis and epididymis were isolated and weighed.

## 2.3. Sperm count

One epididymis of each animal was isolated and cleared of adhering tissues. The head of each epididymis was chopped intosmallpieces(about $1\,\mathrm{mm}^{3}$ )andincubatedin10 mlHam’s F-12 medium at $32\,^{\circ}\mathrm{C}$  for 1 h [7]. Then, 1 ml of the above so- lution was diluted with 9 ml Ham’s F-12 medium. The diluted solution was transferred into each chamber of a haemocy- tometer, and sperm number was counted. To minimize sub- jective error, all operations were performed by one person blinded to the group assignment of animals.

## 2.4. Serum hormone analysis

Serum testosterone concentration was measured by the Roche Elecsys System. The concentrations of LH and FSH were examined by radioimmunoassay, according to the man- ufacture’s instructions. Because commercially available kits are for human serum, the absolute levels of testosterone, FSH and LH could not be qualiﬁed but the comparative levels of these hormone could be ascertained.

## 2.5. Pathological observation

Testicular tissue blocks were ﬁxed in Zenker’s Fluid [8]. Then, samples were sectioned by routine methods followed by H&E staining. Sections were observed by an optical mi- croscope.

## 2.6. TUNEL assay

Testicular tissue blocks were ﬁxed in  $3.7\%$  formaldehyde solution, embedded in parafﬁn and sectioned at $5\,\upmu\mathrm{m}$  for his- tological demonstration of apoptotic fragmentation by the terminal dideoxynucleotidyl transferase dUTP nick end la- beling (TUNEL) method [9,10]. TUNEL-positive cells were counted in 20 randomly selected seminiferous tubules  $(n=$  20).

## 2.7. Cell ultrastructure

Testicular tissue blocks were cut into small pieces, and then ﬁxed in  $2.5\%$  glutaraldehyde. Ultrathin sections were observed by a JEM-1200EX electron microscope.

## 2.8. Statistical analyses

All calculations and statistical analyses were generated in SPSS for windows version 10.0 (SPSS Inc., Chicago, IL, USA). Statistical analyses were performed using one-way analysis of variance (ANOVA) followed by Dunnett’s  $t\mathrm{.}$ -test. Differences were considered to be signiﬁcant at  $P\,<\,0.05$  against control group. Data were presented as mean stan- $\pm$  dard deviation (S.D.).

## 3. Results

There was no signiﬁcant difference in body weight among groups (Table 1). The body weight gain of NP2 group is less than that of the control group, but there is no signiﬁ- cant difference between the two groups. Among rats treated with nonylphenol at  $250\,\mathrm{mg/kg/day}$ , the absolute weight of epididymis decreased dramatically (Table 2). Relative organ weight represents the ratio of organ weight to overall body weight (Table 3). The relative kidney weight of NP2 group increased signiﬁcantly. The relative liver weight increased signiﬁcantly in both NP treated groups. There was no signif- icant difference in relative testis weights among the groups. The relative epididymis weight of the NP2 group dramati- cally decreased. Sperm number in the head of epididymis of the NP2 group dramatically decreased (Table 4).

Serum hormone levels (testosterone, FSH, LH) were mon- itored in male rats and rats exposed to NP (Figs. 1–3). The level of testosterone signiﬁcantly dropped in the NP2 group (Fig. 1). In contrast, FSH (Fig. 2) and LH (Fig. 3) levels in serum were higher in both NP treatment groups versus the control group.

Histopathological examination revealed alterations in the testes of NP1 and NP2 groups (Figs. 4–6). Control animals showed a compact and regular arrangement of cells in the

## 4. Discussion

Nonylphenol, an environmental contaminant, has been reported to induce reproductive abnormalities in male rats [4–6]. In the present study, male SD rats were exposed to  $4–t$ -nonylphenol by gavage at  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2) for 50 days.

A previous study demonstrated that NP was a low-toxic compound, and the $\mathrm{LD}_{50}$  of NP for rat was  $1475\,\mathrm{mg/kg}$  [11]. Several previous researches [4,5] proved that oral exposure of NP has no adverse inﬂuence on the body weight of rats. In our study, the body weight gain of NP2 group is less than that of the control group, but there was no signiﬁcant difference be- tween the two groups. On the other hand, the increases of the relative weights of kidney and liver in NP groups suggested that NP had chronic toxicity to liver and kidney. Chapin’s study also reported that low-dose NP treatment could pose a hazard to the kidney, while no discernable adverse effect for reproductive system were observed [4].

Our study showed that  $250\,\mathrm{mg/kg/day}\,\mathrm{~N~}$ NP induced the descent of the absolute and relative epididymis weight, and decreased the sperm density in the head of epididymis. These results are consistent with several previous studies.

For example, in de Jager’s study, the total cauda epididymal sperm count was signiﬁcantly lower in the  $250\,\mathrm{mg/kg}$  NP dosage group compared to the control group [12]. In another study [5], the absolute weight of the right epididymis signiﬁcantly decreased in rats exposed to 75 or  $15\,\mathrm{mg/kg/day}$  NP. As for the testis, our result is also consistent with many other reports [4,5,13] that all clariﬁed that NP had no effect on the weight of testis.

In mammals, seminiferous epithelium is composed of two basic cell types: spermatogenic cells and Sertoli cells. In our previous study, NP also caused apoptosis of testicular Sertoli cells in vitro [14]. Apoptosis of germ cells is exaggerated after testicular insults and toxicant exposures [15]. In the present study, TUNEL conﬁrmed that NP treatment increased apoptosis of testicular cells (including Sertoli cells and germ cells). It has long been believed that a Sertoli cell will support a number of developing germ cells and this number is quite constant within a species [16]. So, the dysfunction of Sertoli cells and the apoptosis of Sertoli and germ cells may induce the reduction of the sperm production.

Consistent to the result of TUNEL assay, histological changes were observed in the NP groups. The arrangement of cells in the NP groups was irregular and disordered. Inter- cellular connections were incompact. Distinct ultrastructural changes supported the histopathological assessment. Vac- uolization, for example, is common in testicular injury in the interstitial tissue cells and Sertoli cell where apoptosis was induced with deltamethrin [17]. 2,5-Hexanedione exposure also induced an increase in the number and size of Sertoli cell vacuoles [18]. In our experiment, cellular vacuoles increased dramatically in NP1 and NP2 group. Lysosomes play an im- portant role in cell apoptosis during phagocytosis and au- tophagy. Several previous studies have shown that lysosomes increased and clustered during induced apoptosis [19,20]. A similar increase and inﬂation of lysosome were observed here with NP. In addition, we detected an increase of lipofuscin in testicular cells. Lipofuscin accumulation could be regarded as an indicator of oxidative stress [21], and Chitra’s study showed that lipid peroxidation increased signiﬁcantly in the animals treated with nonylphenol [7]. Hence, the lipofuscin accumulation observed here could be attributed to oxidative stress induced by NP treatment.



It has been reported that NP showed an inhibitory effect on P450c17, an important enzyme in testosterone synthesis [22,23].SothedecreaseoftestosteronelevelintheNP2group might result from the inhibition of P450c17. Previous studies demonstrated that secretion of FSH was regulated by inhibin and testosterone through a negative feedback mechanism. In- hibin production of Sertoli cells was inhibited when rats were exposed to NP [24]. So, we deduce that NP ﬁrstly inhibited the inhibin secretion of Sertoli cells and the testosterone se- cretion of Leydig cells, then the decrease of the inhibin and testosterone secretion resulted in the increase of FSH secre- tion. As the secretion of LH was regulated by testosterone in a negative feedback manner, we deduced that down-regulated testosterone levels in NP groups resulted in the excess se- cretion of LH. In Nagao’s study, 8-week-old male rats were treated with $250\,\mathrm{mg/kg/day\,NP}$  for 12 weeks [13]. Decreases of testosterone level and increases of FSH level were ob- served, but the LH level remained unchanged, which was different from our present study.



Thus, the present data show that relative high dose NP can exert a generally chronic toxicity to rats and a particular toxicity to epididymis, induce the apoptosis of testicular cells (including Sertoli cells and germ cells), and indirectly disturb thesecretionsoftestosterone,LHandFSHandtheproduction of sperm.

## References

[1] Giger W, Brunner PH, Schaffner C. 4-Nonylphenol in sewage sludge: accumulation of toxic metabolites from non-ionic surfactants. Sci- ence 1984;225:623–5.

[2] Naylor CG, Mieure JP, Adams WJ, Weeks JA, Castaldi FJ, Ogle LD, et al. Alkylphenol ethoxylates in the environment. J Am Oil Chem Soc 1992;69:695–703.

[3] Ahel M, Mcevoy J, Giger W. Bioaccumulation of the lipophilic metabolites of nonionic surfactants in freshwater organisms. Env- iron Pollut 1993;79:243–8.

[4] Chapin RE, Delaney J, Wang Y, Lanning L, Davis B, Collins B, et al. The effect of 4-nonylphenol in rats: a multigeneration reproduction study. Toxicol Sci 1999;52:80–91.

[5] Hossaini A, Dalgaard M, Vinggaard AM, Frandsen H, Larsen J. In utero reproductive study in rats exposed to nonylphenol. Reprod Toxicol 2001;15:537–43.

[6] Fan Q, Li W, Shen L, Jiang X, Din X. The effects of nonylphenol on the reproductive functions of male SD rats. J Health Toxicol 2001;15:225–6.

[7] Chitra KC, Latchoumycandane C, Mathur PP. Effect of nonylphenol on the antioxidant system in epididymal sperm of rats. Arch Toxicol 2002;76:545–51.

[8] Zhen GC, editor. Microtechnique in biology. Peking, China: People’s Education Press; 1978. p. 7.

[9] Gavrieli Y, Sherman Y, Ben-Sasson SA. Identiﬁcation of pro- grammed cell death in situ via speciﬁc labeling of nuclear DNA fragmentation. J Cell Biol 1992;119:493–501.

[10] Tilly JL, Perez GL. Mechanisms and genes of physiological cell death: a new direction for toxicological risk assessments? Oxford, UK: Elsevier Press; 1997.

[11] de Jager C, Bornman MS, Wandrag S, Sharp VW. Lethal dose and reproductive parameters of  $p$ -nonylphenol in rats. Arch Androl 2001;46:183–7.

[12] de Jager C, Bornman MS, Oosthuizen JM. The effect of  $p$ - nonylphenol on the fertility potential of male rats after gestational, lactational and direct exposure. Andrologia 1999;31:107–13.

[13] Nagao T, Wadaa K, Marumob H, Yoshimurab S, Onoa H. Repro- ductive effects of nonylphenol in rats after gavage administration: a two-generation study. Reprod Toxicol 2001;15:293–315.

[14] Wang X, Han X, Hou Y, Yao G, Wang Y. Effect of nonylphe- nol on apoptosis of Sertoli cells in vitro. Environ Contam Toxicol 2003;70:898–904.

[15] Lee J, Richburg JH, Shipp EB, Meistrich ML, Boekelheide Kim. The Fas system, a regulator of testicular germ cell apoptosis, is differentially up-regulated in Sertoli cell versus germ cell injury of the testis. Endocrinology 1999;140:852–8.

[16] Sharp RM. Regulation of spermatogenesis. In: Knobl E, Neil JD, editors. The physiology of reproduction. New York: Raven Press; p. 1363–434.

[17] El-Gohary M, Awara WM, Nassar S, Hawas S. Deltamethrin-induced testicular apoptosis in rats: the protective effect of nitric oxide syn- thase inhibitor. Toxicology 1999;132:1–8.

[18] Hall ES, Hall SJ, Boekelheide K. 2,5-Hexanedione exposure alters microtubule motor distribution in adult rat testis. Fundam Appl Tox- icol 1995;24:173–82.

[19] Sibayan SAB, Latina MA, Sherwood ME, Flotte TJ, White K. Apop- tosis and morphology changes in drug-treated trabecular meshwork cells in vitro. Exp Eye Res 1998;66:521–9.

[20] Samms M, Philp D, Emanus F, Osuji O, Pezzano M, Guyden JC.Lysosomal-mediateddegradationofapoptoticthymo- cytes within thymic nurse cells. Cell Immunol 1999;197:108– 15.

[21] Sitte N, Merker K, Grune T, Zglinicki T. Lipofuscin accumulation in proliferating ﬁbroblasts in vitro: an indicator of oxidative stress. Exp Gerontol 2001;36:475–86.

[22] Majdic G, Sharpe RM, O’Shaughnessy PJ, Saunders PTK. Expres- sion of cytochrome P45017-hydroxylase/C17-20 lysase in the fetal rat testis in reduced by maternal exposure to exogenous estrogens. Endocrinology 1996;137:1063–70.

[23] Laurenzana EM, Balasubramanian G, Weis C, Blaydes B, New- bold RR, Delclos KB. Effect of nonylphenol on serum testos- terone levels and testicular steroidogenic enzyme activity in neona- tal, pubertal, and adult rats. Chem Biol Interact 2002;139:23– 41.

[24] Fan Q, Jiang X, Ding X, Jin T. The effects of nonylphenol on inhibin and androgen binding protein expressions in the testis of rats. Acta Acad Med Zunyi 2002;25:197–201.
# The toxic effects of nonylphenol on the reproductive system of male rats

Xiao D. Han,∗, Zhi G. Tu, Yi Gong, Su N. Shen, Xu Y. Wang, Li N. Kang, Ya Y. Hou, Jian X. Chen

 Immunology and Reproductive Biology Laboratory, Medical School, Nanjing University, Nanjing, Jiangsu 210093, PR China  Department of Biology, School of Life Science, Nanjing University, Nanjing, Jiangsu 210093, PR China Received 20 November 2003; received in revised form 16 April 2004; accepted 25 June 2004 Available online 19 August 2004



### Abstract

Male Sprague–Dawley (SD) rats were exposed to $^{4-t}$ -nonylphenol (NP) by gavage at dosages of 0, 125 and  $250\,\mathrm{mg/kg/day}$  for 50 days. Organ weights of liver, kidney, testis and epididymis were measured. Sperm number in the head of epididymis was counted. Several hormones including testosterone, luteinizing hormone and follicle stimulating hormone were measured. Testicular sections were observed by light and electron microscopy. Terminal dideoxynucleotidyl transferase dUTP nick end labeling assay was performed to probe the apoptotic cells in seminiferous tubules. When rats were treated with nonylphenol at  $250\,\mathrm{mg/kg/day},$ , the absolute and relative weight of epididymis decreased dramatically, while the relative weights of kidney and liver increased by 14 and $22\%$ , respectively. In addition, the sperm density of the head of epididymis and the testosterone level descended at  $250\,\mathrm{mg/kg/day}$ . The levels of luteinizing hormone and follicle stimulating hormone increased in both nonylphenol treated groups. Pathological changes were detected by microscopy and the transferase dUTP nick end labeling (TUNEL) assay showed that the number of apoptotic cells in testes increased with nonylphenol in a dose-dependent manner.

 $\copyright2004$  Elsevier Inc. All rights reserved.

Keywords: Nonylphonol; Apoptosis; Reproductive system; Testosterone; Luteinizing hormone; Follicle stimulating hormone; TUNEL

## 1. Introduction

Nonylphenol polyethoxylates (NPEs) are non-ionic sur- factants that are widely used as components of detergents, paints, herbicides, insecticides and many other synthetic products. With their wide use, a large amount of NPEs are charged into water. Nonylphenol (NP), a primary ﬁnal biodegradation product of NPEs [1], has been documented to appear in aquatic environment, particularly in sediment. Its content may reach  $3000\;\mathrm{ppb}$  in rivers and lakes [2]. NP showed a great capacity of bioaccumulation in aquatic an- imals, and investigators found that the NP concentration in the muscle of Anas boscas (living in contaminated aquatic environment) reached  $1.2\,\mathrm{mg/kg}$  [3].

NP has been reported to be weakly estrogenic from in vitro and in vivo studies. In Chapin’s study [4], 3.5 generations of rats were exposed to NP in the diet at 200, 650 and $2000\,\mathrm{ppm}$ , which corresponded to dietary intakes of 9–35, 30–100, and  $100{-}350\,\mathrm{mg/kg/day}$ . Vaginal opening was accelerated by 2 days (at  $650\,\mathrm{ppm})$  and 6 days (at $2000\,\mathrm{ppm})$ ). Uterine weights at postnatal day 21 were increased in F1 females treated with 650 and 2000 ppm NP. In F2 rats, epididymal sperm density was reduced by 8 and  $13\%$  at 650 and 2000 ppm, respec- tively. Collectively, these results suggest that NP has weak estrogenic activity and may induce abnormities in female and male reproductive systems. In Hossaini’s study [5], female pregnant rats were dosed with NP at 3, 15, or  $75\,\mathrm{mg/kg/c}$ day from gestation days 11 to 18. Male offsprings were sacri- ﬁced at postnatal days 11, 21, and 110, and their reproduc- tive parameters were evaluated. The absolute weight of the right epididymis signiﬁcantly decreased in the animals whose mother was exposed to 75 or 15 mg/kg/day NP. In another study,  $100\,\mathrm{mg/kg/day}\,\mathrm{N}$ P reduced the daily sperm production of male rats, and  $300\,\mathrm{mg/kg/day}\ \mathrm{N}$ P induced the atrophy of testis and epididymis [6]. Until now, systemic studies on the male reproductive toxicity of NP have been scarce. The aim of this present study is to further investigate the mechanisms of the reproductive toxicity of NP in male rats.



## 2. Materials and methods

## 2.1. Instruments and reagents

Instruments: Roche Elecsys System (2010), Electron microscope (JEM-1200EX),  $\upgamma$ -counter (SN-695). Sources of reagents were as follows: corn oil (Fortune, Shang- hai, PR China); technical nonylphenol (FGJ 01, purity:  $98\%$ , Tokyo Kasei Kogyo Co. Ltd.); pentobarbital (Sigma, USA); testosterone reagent kit (Roche Diagnostics GmbH, D-68298 Mannheim, Germany); human luteinizing hor- mone radioimmunoassay kit and follicle stimulating hor- mone radioimmunoassay kit (Jiuding Co. Ltd., PR China); apoptosis detection kit (R&D systems, Minneapolis, MN, USA).

## 2.2. Animal treatments

Thirty male Sprague–Dawley rats, aging 28–30 days, were purchased from Nanjing Medical University and housed in groups at  $24\,^{\circ}\mathrm{C}$  in a humidity-controlled room with 12 h light/dark photoperiod. Standard rat chow was ob- tained from Nanjing Medical University (Nanjing Medical University, Nanjing, P.R. China) and available throughout the experiment. Rats were acclimated for 1 week prior to treatments.

Male rats were divided into three groups: control, NP1 and NP2. NP was dissolved in corn oil and given to rats by gavage at  $0\,\mathrm{mg/kg/day}$  (control),  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2). Gavage volume was adjusted accord- ing to the weight of each rat. During the experiment, one rat in the NP2 group manifested a marked reduction in body weight gain and was excluded from the formal experiment. After 50 days treatment, the rats were weighed and anaesthetized by injecting pentobarbital. So far, no report indicates that pen- tobarbital can affect the levels of LH, FSH and testosterone. To obtain pure serum, blood samples were collected via the carotid artery. We inserted a ﬁne tube into the carotid to draw blood into a test tube. When the content of extracted blood reached $3.5–4.5\%$  body weight, this operation was regarded as successful; else it was designated a failure. Because the residual blood in the body will affect organ weight, the cases called failure were excluded from the control, NP1 and NP2 groups as follows: 2, 2 and 1, cases, respectively. At the end of this study, each group had eight rats that can offer available data. After this step, liver, kidney, testis and epididymis were isolated and weighed.

## 2.3. Sperm count

One epididymis of each animal was isolated and cleared of adhering tissues. The head of each epididymis was chopped intosmallpieces(about $1\,\mathrm{mm}^{3}$ )andincubatedin10 mlHam’s F-12 medium at $32\,^{\circ}\mathrm{C}$  for 1 h [7]. Then, 1 ml of the above so- lution was diluted with 9 ml Ham’s F-12 medium. The diluted solution was transferred into each chamber of a haemocy- tometer, and sperm number was counted. To minimize sub- jective error, all operations were performed by one person blinded to the group assignment of animals.

## 2.4. Serum hormone analysis

Serum testosterone concentration was measured by the Roche Elecsys System. The concentrations of LH and FSH were examined by radioimmunoassay, according to the man- ufacture’s instructions. Because commercially available kits are for human serum, the absolute levels of testosterone, FSH and LH could not be qualiﬁed but the comparative levels of these hormone could be ascertained.

## 2.5. Pathological observation

Testicular tissue blocks were ﬁxed in Zenker’s Fluid [8]. Then, samples were sectioned by routine methods followed by H&E staining. Sections were observed by an optical mi- croscope.

## 2.6. TUNEL assay

Testicular tissue blocks were ﬁxed in  $3.7\%$  formaldehyde solution, embedded in parafﬁn and sectioned at $5\,\upmu\mathrm{m}$  for his- tological demonstration of apoptotic fragmentation by the terminal dideoxynucleotidyl transferase dUTP nick end la- beling (TUNEL) method [9,10]. TUNEL-positive cells were counted in 20 randomly selected seminiferous tubules  $(n=$  20).

## 2.7. Cell ultrastructure

Testicular tissue blocks were cut into small pieces, and then ﬁxed in  $2.5\%$  glutaraldehyde. Ultrathin sections were observed by a JEM-1200EX electron microscope.

## 2.8. Statistical analyses

All calculations and statistical analyses were generated in SPSS for windows version 10.0 (SPSS Inc., Chicago, IL, USA). Statistical analyses were performed using one-way analysis of variance (ANOVA) followed by Dunnett’s  $t\mathrm{.}$ -test. Differences were considered to be signiﬁcant at  $P\,<\,0.05$  against control group. Data were presented as mean stan- $\pm$  dard deviation (S.D.).

## 3. Results

There was no signiﬁcant difference in body weight among groups (Table 1). The body weight gain of NP2 group is less than that of the control group, but there is no signiﬁ- cant difference between the two groups. Among rats treated with nonylphenol at  $250\,\mathrm{mg/kg/day}$ , the absolute weight of epididymis decreased dramatically (Table 2). Relative organ weight represents the ratio of organ weight to overall body weight (Table 3). The relative kidney weight of NP2 group increased signiﬁcantly. The relative liver weight increased signiﬁcantly in both NP treated groups. There was no signif- icant difference in relative testis weights among the groups. The relative epididymis weight of the NP2 group dramati- cally decreased. Sperm number in the head of epididymis of the NP2 group dramatically decreased (Table 4).

Serum hormone levels (testosterone, FSH, LH) were mon- itored in male rats and rats exposed to NP (Figs. 1–3). The level of testosterone signiﬁcantly dropped in the NP2 group (Fig. 1). In contrast, FSH (Fig. 2) and LH (Fig. 3) levels in serum were higher in both NP treatment groups versus the control group.

Histopathological examination revealed alterations in the testes of NP1 and NP2 groups (Figs. 4–6). Control animals showed a compact and regular arrangement of cells in the

## 4. Discussion

Nonylphenol, an environmental contaminant, has been reported to induce reproductive abnormalities in male rats [4–6]. In the present study, male SD rats were exposed to  $4–t$ -nonylphenol by gavage at  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2) for 50 days.

A previous study demonstrated that NP was a low-toxic compound, and the $\mathrm{LD}_{50}$  of NP for rat was  $1475\,\mathrm{mg/kg}$  [11]. Several previous researches [4,5] proved that oral exposure of NP has no adverse inﬂuence on the body weight of rats. In our study, the body weight gain of NP2 group is less than that of the control group, but there was no signiﬁcant difference be- tween the two groups. On the other hand, the increases of the relative weights of kidney and liver in NP groups suggested that NP had chronic toxicity to liver and kidney. Chapin’s study also reported that low-dose NP treatment could pose a hazard to the kidney, while no discernable adverse effect for reproductive system were observed [4].

Our study showed that  $250\,\mathrm{mg/kg/day}\,\mathrm{~N~}$ NP induced the descent of the absolute and relative epididymis weight, and decreased the sperm density in the head of epididymis. These results are consistent with several previous studies.

For example, in de Jager’s study, the total cauda epididymal sperm count was signiﬁcantly lower in the  $250\,\mathrm{mg/kg}$  NP dosage group compared to the control group [12]. In another study [5], the absolute weight of the right epididymis signiﬁcantly decreased in rats exposed to 75 or  $15\,\mathrm{mg/kg/day}$  NP. As for the testis, our result is also consistent with many other reports [4,5,13] that all clariﬁed that NP had no effect on the weight of testis.

In mammals, seminiferous epithelium is composed of two basic cell types: spermatogenic cells and Sertoli cells. In our previous study, NP also caused apoptosis of testicular Sertoli cells in vitro [14]. Apoptosis of germ cells is exaggerated after testicular insults and toxicant exposures [15]. In the present study, TUNEL conﬁrmed that NP treatment increased apoptosis of testicular cells (including Sertoli cells and germ cells). It has long been believed that a Sertoli cell will support a number of developing germ cells and this number is quite constant within a species [16]. So, the dysfunction of Sertoli cells and the apoptosis of Sertoli and germ cells may induce the reduction of the sperm production.

Consistent to the result of TUNEL assay, histological changes were observed in the NP groups. The arrangement of cells in the NP groups was irregular and disordered. Inter- cellular connections were incompact. Distinct ultrastructural changes supported the histopathological assessment. Vac- uolization, for example, is common in testicular injury in the interstitial tissue cells and Sertoli cell where apoptosis was induced with deltamethrin [17]. 2,5-Hexanedione exposure also induced an increase in the number and size of Sertoli cell vacuoles [18]. In our experiment, cellular vacuoles increased dramatically in NP1 and NP2 group. Lysosomes play an im- portant role in cell apoptosis during phagocytosis and au- tophagy. Several previous studies have shown that lysosomes increased and clustered during induced apoptosis [19,20]. A similar increase and inﬂation of lysosome were observed here with NP. In addition, we detected an increase of lipofuscin in testicular cells. Lipofuscin accumulation could be regarded as an indicator of oxidative stress [21], and Chitra’s study showed that lipid peroxidation increased signiﬁcantly in the animals treated with nonylphenol [7]. Hence, the lipofuscin accumulation observed here could be attributed to oxidative stress induced by NP treatment.



It has been reported that NP showed an inhibitory effect on P450c17, an important enzyme in testosterone synthesis [22,23].SothedecreaseoftestosteronelevelintheNP2group might result from the inhibition of P450c17. Previous studies demonstrated that secretion of FSH was regulated by inhibin and testosterone through a negative feedback mechanism. In- hibin production of Sertoli cells was inhibited when rats were exposed to NP [24]. So, we deduce that NP ﬁrstly inhibited the inhibin secretion of Sertoli cells and the testosterone se- cretion of Leydig cells, then the decrease of the inhibin and testosterone secretion resulted in the increase of FSH secre- tion. As the secretion of LH was regulated by testosterone in a negative feedback manner, we deduced that down-regulated testosterone levels in NP groups resulted in the excess se- cretion of LH. In Nagao’s study, 8-week-old male rats were treated with $250\,\mathrm{mg/kg/day\,NP}$  for 12 weeks [13]. Decreases of testosterone level and increases of FSH level were ob- served, but the LH level remained unchanged, which was different from our present study.



Thus, the present data show that relative high dose NP can exert a generally chronic toxicity to rats and a particular toxicity to epididymis, induce the apoptosis of testicular cells (including Sertoli cells and germ cells), and indirectly disturb thesecretionsoftestosterone,LHandFSHandtheproduction of sperm.

## References

[1] Giger W, Brunner PH, Schaffner C. 4-Nonylphenol in sewage sludge: accumulation of toxic metabolites from non-ionic surfactants. Sci- ence 1984;225:623–5.

[2] Naylor CG, Mieure JP, Adams WJ, Weeks JA, Castaldi FJ, Ogle LD, et al. Alkylphenol ethoxylates in the environment. J Am Oil Chem Soc 1992;69:695–703.

[3] Ahel M, Mcevoy J, Giger W. Bioaccumulation of the lipophilic metabolites of nonionic surfactants in freshwater organisms. Env- iron Pollut 1993;79:243–8.

[4] Chapin RE, Delaney J, Wang Y, Lanning L, Davis B, Collins B, et al. The effect of 4-nonylphenol in rats: a multigeneration reproduction study. Toxicol Sci 1999;52:80–91.

[5] Hossaini A, Dalgaard M, Vinggaard AM, Frandsen H, Larsen J. In utero reproductive study in rats exposed to nonylphenol. Reprod Toxicol 2001;15:537–43.

[6] Fan Q, Li W, Shen L, Jiang X, Din X. The effects of nonylphenol on the reproductive functions of male SD rats. J Health Toxicol 2001;15:225–6.

[7] Chitra KC, Latchoumycandane C, Mathur PP. Effect of nonylphenol on the antioxidant system in epididymal sperm of rats. Arch Toxicol 2002;76:545–51.

[8] Zhen GC, editor. Microtechnique in biology. Peking, China: People’s Education Press; 1978. p. 7.

[9] Gavrieli Y, Sherman Y, Ben-Sasson SA. Identiﬁcation of pro- grammed cell death in situ via speciﬁc labeling of nuclear DNA fragmentation. J Cell Biol 1992;119:493–501.

[10] Tilly JL, Perez GL. Mechanisms and genes of physiological cell death: a new direction for toxicological risk assessments? Oxford, UK: Elsevier Press; 1997.

[11] de Jager C, Bornman MS, Wandrag S, Sharp VW. Lethal dose and reproductive parameters of  $p$ -nonylphenol in rats. Arch Androl 2001;46:183–7.

[12] de Jager C, Bornman MS, Oosthuizen JM. The effect of  $p$ - nonylphenol on the fertility potential of male rats after gestational, lactational and direct exposure. Andrologia 1999;31:107–13.

[13] Nagao T, Wadaa K, Marumob H, Yoshimurab S, Onoa H. Repro- ductive effects of nonylphenol in rats after gavage administration: a two-generation study. Reprod Toxicol 2001;15:293–315.

[14] Wang X, Han X, Hou Y, Yao G, Wang Y. Effect of nonylphe- nol on apoptosis of Sertoli cells in vitro. Environ Contam Toxicol 2003;70:898–904.

[15] Lee J, Richburg JH, Shipp EB, Meistrich ML, Boekelheide Kim. The Fas system, a regulator of testicular germ cell apoptosis, is differentially up-regulated in Sertoli cell versus germ cell injury of the testis. Endocrinology 1999;140:852–8.

[16] Sharp RM. Regulation of spermatogenesis. In: Knobl E, Neil JD, editors. The physiology of reproduction. New York: Raven Press; p. 1363–434.

[17] El-Gohary M, Awara WM, Nassar S, Hawas S. Deltamethrin-induced testicular apoptosis in rats: the protective effect of nitric oxide syn- thase inhibitor. Toxicology 1999;132:1–8.

[18] Hall ES, Hall SJ, Boekelheide K. 2,5-Hexanedione exposure alters microtubule motor distribution in adult rat testis. Fundam Appl Tox- icol 1995;24:173–82.

[19] Sibayan SAB, Latina MA, Sherwood ME, Flotte TJ, White K. Apop- tosis and morphology changes in drug-treated trabecular meshwork cells in vitro. Exp Eye Res 1998;66:521–9.

[20] Samms M, Philp D, Emanus F, Osuji O, Pezzano M, Guyden JC.Lysosomal-mediateddegradationofapoptoticthymo- cytes within thymic nurse cells. Cell Immunol 1999;197:108– 15.

[21] Sitte N, Merker K, Grune T, Zglinicki T. Lipofuscin accumulation in proliferating ﬁbroblasts in vitro: an indicator of oxidative stress. Exp Gerontol 2001;36:475–86.

[22] Majdic G, Sharpe RM, O’Shaughnessy PJ, Saunders PTK. Expres- sion of cytochrome P45017-hydroxylase/C17-20 lysase in the fetal rat testis in reduced by maternal exposure to exogenous estrogens. Endocrinology 1996;137:1063–70.

[23] Laurenzana EM, Balasubramanian G, Weis C, Blaydes B, New- bold RR, Delclos KB. Effect of nonylphenol on serum testos- terone levels and testicular steroidogenic enzyme activity in neona- tal, pubertal, and adult rats. Chem Biol Interact 2002;139:23– 41.

[24] Fan Q, Jiang X, Ding X, Jin T. The effects of nonylphenol on inhibin and androgen binding protein expressions in the testis of rats. Acta Acad Med Zunyi 2002;25:197–201.
# The toxic effects of nonylphenol on the reproductive system of male rats

Xiao D. Han,∗, Zhi G. Tu, Yi Gong, Su N. Shen, Xu Y. Wang, Li N. Kang, Ya Y. Hou, Jian X. Chen

 Immunology and Reproductive Biology Laboratory, Medical School, Nanjing University, Nanjing, Jiangsu 210093, PR China  Department of Biology, School of Life Science, Nanjing University, Nanjing, Jiangsu 210093, PR China Received 20 November 2003; received in revised form 16 April 2004; accepted 25 June 2004 Available online 19 August 2004



### Abstract

Male Sprague–Dawley (SD) rats were exposed to $^{4-t}$ -nonylphenol (NP) by gavage at dosages of 0, 125 and  $250\,\mathrm{mg/kg/day}$  for 50 days. Organ weights of liver, kidney, testis and epididymis were measured. Sperm number in the head of epididymis was counted. Several hormones including testosterone, luteinizing hormone and follicle stimulating hormone were measured. Testicular sections were observed by light and electron microscopy. Terminal dideoxynucleotidyl transferase dUTP nick end labeling assay was performed to probe the apoptotic cells in seminiferous tubules. When rats were treated with nonylphenol at  $250\,\mathrm{mg/kg/day},$ , the absolute and relative weight of epididymis decreased dramatically, while the relative weights of kidney and liver increased by 14 and $22\%$ , respectively. In addition, the sperm density of the head of epididymis and the testosterone level descended at  $250\,\mathrm{mg/kg/day}$ . The levels of luteinizing hormone and follicle stimulating hormone increased in both nonylphenol treated groups. Pathological changes were detected by microscopy and the transferase dUTP nick end labeling (TUNEL) assay showed that the number of apoptotic cells in testes increased with nonylphenol in a dose-dependent manner.

 $\copyright2004$  Elsevier Inc. All rights reserved.

Keywords: Nonylphonol; Apoptosis; Reproductive system; Testosterone; Luteinizing hormone; Follicle stimulating hormone; TUNEL

## 1. Introduction

Nonylphenol polyethoxylates (NPEs) are non-ionic sur- factants that are widely used as components of detergents, paints, herbicides, insecticides and many other synthetic products. With their wide use, a large amount of NPEs are charged into water. Nonylphenol (NP), a primary ﬁnal biodegradation product of NPEs [1], has been documented to appear in aquatic environment, particularly in sediment. Its content may reach  $3000\;\mathrm{ppb}$  in rivers and lakes [2]. NP showed a great capacity of bioaccumulation in aquatic an- imals, and investigators found that the NP concentration in the muscle of Anas boscas (living in contaminated aquatic environment) reached  $1.2\,\mathrm{mg/kg}$  [3].

NP has been reported to be weakly estrogenic from in vitro and in vivo studies. In Chapin’s study [4], 3.5 generations of rats were exposed to NP in the diet at 200, 650 and $2000\,\mathrm{ppm}$ , which corresponded to dietary intakes of 9–35, 30–100, and  $100{-}350\,\mathrm{mg/kg/day}$ . Vaginal opening was accelerated by 2 days (at  $650\,\mathrm{ppm})$  and 6 days (at $2000\,\mathrm{ppm})$ ). Uterine weights at postnatal day 21 were increased in F1 females treated with 650 and 2000 ppm NP. In F2 rats, epididymal sperm density was reduced by 8 and  $13\%$  at 650 and 2000 ppm, respec- tively. Collectively, these results suggest that NP has weak estrogenic activity and may induce abnormities in female and male reproductive systems. In Hossaini’s study [5], female pregnant rats were dosed with NP at 3, 15, or  $75\,\mathrm{mg/kg/c}$ day from gestation days 11 to 18. Male offsprings were sacri- ﬁced at postnatal days 11, 21, and 110, and their reproduc- tive parameters were evaluated. The absolute weight of the right epididymis signiﬁcantly decreased in the animals whose mother was exposed to 75 or 15 mg/kg/day NP. In another study,  $100\,\mathrm{mg/kg/day}\,\mathrm{N}$ P reduced the daily sperm production of male rats, and  $300\,\mathrm{mg/kg/day}\ \mathrm{N}$ P induced the atrophy of testis and epididymis [6]. Until now, systemic studies on the male reproductive toxicity of NP have been scarce. The aim of this present study is to further investigate the mechanisms of the reproductive toxicity of NP in male rats.



## 2. Materials and methods

## 2.1. Instruments and reagents

Instruments: Roche Elecsys System (2010), Electron microscope (JEM-1200EX),  $\upgamma$ -counter (SN-695). Sources of reagents were as follows: corn oil (Fortune, Shang- hai, PR China); technical nonylphenol (FGJ 01, purity:  $98\%$ , Tokyo Kasei Kogyo Co. Ltd.); pentobarbital (Sigma, USA); testosterone reagent kit (Roche Diagnostics GmbH, D-68298 Mannheim, Germany); human luteinizing hor- mone radioimmunoassay kit and follicle stimulating hor- mone radioimmunoassay kit (Jiuding Co. Ltd., PR China); apoptosis detection kit (R&D systems, Minneapolis, MN, USA).

## 2.2. Animal treatments

Thirty male Sprague–Dawley rats, aging 28–30 days, were purchased from Nanjing Medical University and housed in groups at  $24\,^{\circ}\mathrm{C}$  in a humidity-controlled room with 12 h light/dark photoperiod. Standard rat chow was ob- tained from Nanjing Medical University (Nanjing Medical University, Nanjing, P.R. China) and available throughout the experiment. Rats were acclimated for 1 week prior to treatments.

Male rats were divided into three groups: control, NP1 and NP2. NP was dissolved in corn oil and given to rats by gavage at  $0\,\mathrm{mg/kg/day}$  (control),  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2). Gavage volume was adjusted accord- ing to the weight of each rat. During the experiment, one rat in the NP2 group manifested a marked reduction in body weight gain and was excluded from the formal experiment. After 50 days treatment, the rats were weighed and anaesthetized by injecting pentobarbital. So far, no report indicates that pen- tobarbital can affect the levels of LH, FSH and testosterone. To obtain pure serum, blood samples were collected via the carotid artery. We inserted a ﬁne tube into the carotid to draw blood into a test tube. When the content of extracted blood reached $3.5–4.5\%$  body weight, this operation was regarded as successful; else it was designated a failure. Because the residual blood in the body will affect organ weight, the cases called failure were excluded from the control, NP1 and NP2 groups as follows: 2, 2 and 1, cases, respectively. At the end of this study, each group had eight rats that can offer available data. After this step, liver, kidney, testis and epididymis were isolated and weighed.

## 2.3. Sperm count

One epididymis of each animal was isolated and cleared of adhering tissues. The head of each epididymis was chopped intosmallpieces(about $1\,\mathrm{mm}^{3}$ )andincubatedin10 mlHam’s F-12 medium at $32\,^{\circ}\mathrm{C}$  for 1 h [7]. Then, 1 ml of the above so- lution was diluted with 9 ml Ham’s F-12 medium. The diluted solution was transferred into each chamber of a haemocy- tometer, and sperm number was counted. To minimize sub- jective error, all operations were performed by one person blinded to the group assignment of animals.

## 2.4. Serum hormone analysis

Serum testosterone concentration was measured by the Roche Elecsys System. The concentrations of LH and FSH were examined by radioimmunoassay, according to the man- ufacture’s instructions. Because commercially available kits are for human serum, the absolute levels of testosterone, FSH and LH could not be qualiﬁed but the comparative levels of these hormone could be ascertained.

## 2.5. Pathological observation

Testicular tissue blocks were ﬁxed in Zenker’s Fluid [8]. Then, samples were sectioned by routine methods followed by H&E staining. Sections were observed by an optical mi- croscope.

## 2.6. TUNEL assay

Testicular tissue blocks were ﬁxed in  $3.7\%$  formaldehyde solution, embedded in parafﬁn and sectioned at $5\,\upmu\mathrm{m}$  for his- tological demonstration of apoptotic fragmentation by the terminal dideoxynucleotidyl transferase dUTP nick end la- beling (TUNEL) method [9,10]. TUNEL-positive cells were counted in 20 randomly selected seminiferous tubules  $(n=$  20).

## 2.7. Cell ultrastructure

Testicular tissue blocks were cut into small pieces, and then ﬁxed in  $2.5\%$  glutaraldehyde. Ultrathin sections were observed by a JEM-1200EX electron microscope.

## 2.8. Statistical analyses

All calculations and statistical analyses were generated in SPSS for windows version 10.0 (SPSS Inc., Chicago, IL, USA). Statistical analyses were performed using one-way analysis of variance (ANOVA) followed by Dunnett’s  $t\mathrm{.}$ -test. Differences were considered to be signiﬁcant at  $P\,<\,0.05$  against control group. Data were presented as mean stan- $\pm$  dard deviation (S.D.).

## 3. Results

There was no signiﬁcant difference in body weight among groups (Table 1). The body weight gain of NP2 group is less than that of the control group, but there is no signiﬁ- cant difference between the two groups. Among rats treated with nonylphenol at  $250\,\mathrm{mg/kg/day}$ , the absolute weight of epididymis decreased dramatically (Table 2). Relative organ weight represents the ratio of organ weight to overall body weight (Table 3). The relative kidney weight of NP2 group increased signiﬁcantly. The relative liver weight increased signiﬁcantly in both NP treated groups. There was no signif- icant difference in relative testis weights among the groups. The relative epididymis weight of the NP2 group dramati- cally decreased. Sperm number in the head of epididymis of the NP2 group dramatically decreased (Table 4).

Serum hormone levels (testosterone, FSH, LH) were mon- itored in male rats and rats exposed to NP (Figs. 1–3). The level of testosterone signiﬁcantly dropped in the NP2 group (Fig. 1). In contrast, FSH (Fig. 2) and LH (Fig. 3) levels in serum were higher in both NP treatment groups versus the control group.

Histopathological examination revealed alterations in the testes of NP1 and NP2 groups (Figs. 4–6). Control animals showed a compact and regular arrangement of cells in the

## 4. Discussion

Nonylphenol, an environmental contaminant, has been reported to induce reproductive abnormalities in male rats [4–6]. In the present study, male SD rats were exposed to  $4–t$ -nonylphenol by gavage at  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2) for 50 days.

A previous study demonstrated that NP was a low-toxic compound, and the $\mathrm{LD}_{50}$  of NP for rat was  $1475\,\mathrm{mg/kg}$  [11]. Several previous researches [4,5] proved that oral exposure of NP has no adverse inﬂuence on the body weight of rats. In our study, the body weight gain of NP2 group is less than that of the control group, but there was no signiﬁcant difference be- tween the two groups. On the other hand, the increases of the relative weights of kidney and liver in NP groups suggested that NP had chronic toxicity to liver and kidney. Chapin’s study also reported that low-dose NP treatment could pose a hazard to the kidney, while no discernable adverse effect for reproductive system were observed [4].

Our study showed that  $250\,\mathrm{mg/kg/day}\,\mathrm{~N~}$ NP induced the descent of the absolute and relative epididymis weight, and decreased the sperm density in the head of epididymis. These results are consistent with several previous studies.

For example, in de Jager’s study, the total cauda epididymal sperm count was signiﬁcantly lower in the  $250\,\mathrm{mg/kg}$  NP dosage group compared to the control group [12]. In another study [5], the absolute weight of the right epididymis signiﬁcantly decreased in rats exposed to 75 or  $15\,\mathrm{mg/kg/day}$  NP. As for the testis, our result is also consistent with many other reports [4,5,13] that all clariﬁed that NP had no effect on the weight of testis.

In mammals, seminiferous epithelium is composed of two basic cell types: spermatogenic cells and Sertoli cells. In our previous study, NP also caused apoptosis of testicular Sertoli cells in vitro [14]. Apoptosis of germ cells is exaggerated after testicular insults and toxicant exposures [15]. In the present study, TUNEL conﬁrmed that NP treatment increased apoptosis of testicular cells (including Sertoli cells and germ cells). It has long been believed that a Sertoli cell will support a number of developing germ cells and this number is quite constant within a species [16]. So, the dysfunction of Sertoli cells and the apoptosis of Sertoli and germ cells may induce the reduction of the sperm production.

Consistent to the result of TUNEL assay, histological changes were observed in the NP groups. The arrangement of cells in the NP groups was irregular and disordered. Inter- cellular connections were incompact. Distinct ultrastructural changes supported the histopathological assessment. Vac- uolization, for example, is common in testicular injury in the interstitial tissue cells and Sertoli cell where apoptosis was induced with deltamethrin [17]. 2,5-Hexanedione exposure also induced an increase in the number and size of Sertoli cell vacuoles [18]. In our experiment, cellular vacuoles increased dramatically in NP1 and NP2 group. Lysosomes play an im- portant role in cell apoptosis during phagocytosis and au- tophagy. Several previous studies have shown that lysosomes increased and clustered during induced apoptosis [19,20]. A similar increase and inﬂation of lysosome were observed here with NP. In addition, we detected an increase of lipofuscin in testicular cells. Lipofuscin accumulation could be regarded as an indicator of oxidative stress [21], and Chitra’s study showed that lipid peroxidation increased signiﬁcantly in the animals treated with nonylphenol [7]. Hence, the lipofuscin accumulation observed here could be attributed to oxidative stress induced by NP treatment.



It has been reported that NP showed an inhibitory effect on P450c17, an important enzyme in testosterone synthesis [22,23].SothedecreaseoftestosteronelevelintheNP2group might result from the inhibition of P450c17. Previous studies demonstrated that secretion of FSH was regulated by inhibin and testosterone through a negative feedback mechanism. In- hibin production of Sertoli cells was inhibited when rats were exposed to NP [24]. So, we deduce that NP ﬁrstly inhibited the inhibin secretion of Sertoli cells and the testosterone se- cretion of Leydig cells, then the decrease of the inhibin and testosterone secretion resulted in the increase of FSH secre- tion. As the secretion of LH was regulated by testosterone in a negative feedback manner, we deduced that down-regulated testosterone levels in NP groups resulted in the excess se- cretion of LH. In Nagao’s study, 8-week-old male rats were treated with $250\,\mathrm{mg/kg/day\,NP}$  for 12 weeks [13]. Decreases of testosterone level and increases of FSH level were ob- served, but the LH level remained unchanged, which was different from our present study.



Thus, the present data show that relative high dose NP can exert a generally chronic toxicity to rats and a particular toxicity to epididymis, induce the apoptosis of testicular cells (including Sertoli cells and germ cells), and indirectly disturb thesecretionsoftestosterone,LHandFSHandtheproduction of sperm.

## References

[1] Giger W, Brunner PH, Schaffner C. 4-Nonylphenol in sewage sludge: accumulation of toxic metabolites from non-ionic surfactants. Sci- ence 1984;225:623–5.

[2] Naylor CG, Mieure JP, Adams WJ, Weeks JA, Castaldi FJ, Ogle LD, et al. Alkylphenol ethoxylates in the environment. J Am Oil Chem Soc 1992;69:695–703.

[3] Ahel M, Mcevoy J, Giger W. Bioaccumulation of the lipophilic metabolites of nonionic surfactants in freshwater organisms. Env- iron Pollut 1993;79:243–8.

[4] Chapin RE, Delaney J, Wang Y, Lanning L, Davis B, Collins B, et al. The effect of 4-nonylphenol in rats: a multigeneration reproduction study. Toxicol Sci 1999;52:80–91.

[5] Hossaini A, Dalgaard M, Vinggaard AM, Frandsen H, Larsen J. In utero reproductive study in rats exposed to nonylphenol. Reprod Toxicol 2001;15:537–43.

[6] Fan Q, Li W, Shen L, Jiang X, Din X. The effects of nonylphenol on the reproductive functions of male SD rats. J Health Toxicol 2001;15:225–6.

[7] Chitra KC, Latchoumycandane C, Mathur PP. Effect of nonylphenol on the antioxidant system in epididymal sperm of rats. Arch Toxicol 2002;76:545–51.

[8] Zhen GC, editor. Microtechnique in biology. Peking, China: People’s Education Press; 1978. p. 7.

[9] Gavrieli Y, Sherman Y, Ben-Sasson SA. Identiﬁcation of pro- grammed cell death in situ via speciﬁc labeling of nuclear DNA fragmentation. J Cell Biol 1992;119:493–501.

[10] Tilly JL, Perez GL. Mechanisms and genes of physiological cell death: a new direction for toxicological risk assessments? Oxford, UK: Elsevier Press; 1997.

[11] de Jager C, Bornman MS, Wandrag S, Sharp VW. Lethal dose and reproductive parameters of  $p$ -nonylphenol in rats. Arch Androl 2001;46:183–7.

[12] de Jager C, Bornman MS, Oosthuizen JM. The effect of  $p$ - nonylphenol on the fertility potential of male rats after gestational, lactational and direct exposure. Andrologia 1999;31:107–13.

[13] Nagao T, Wadaa K, Marumob H, Yoshimurab S, Onoa H. Repro- ductive effects of nonylphenol in rats after gavage administration: a two-generation study. Reprod Toxicol 2001;15:293–315.

[14] Wang X, Han X, Hou Y, Yao G, Wang Y. Effect of nonylphe- nol on apoptosis of Sertoli cells in vitro. Environ Contam Toxicol 2003;70:898–904.

[15] Lee J, Richburg JH, Shipp EB, Meistrich ML, Boekelheide Kim. The Fas system, a regulator of testicular germ cell apoptosis, is differentially up-regulated in Sertoli cell versus germ cell injury of the testis. Endocrinology 1999;140:852–8.

[16] Sharp RM. Regulation of spermatogenesis. In: Knobl E, Neil JD, editors. The physiology of reproduction. New York: Raven Press; p. 1363–434.

[17] El-Gohary M, Awara WM, Nassar S, Hawas S. Deltamethrin-induced testicular apoptosis in rats: the protective effect of nitric oxide syn- thase inhibitor. Toxicology 1999;132:1–8.

[18] Hall ES, Hall SJ, Boekelheide K. 2,5-Hexanedione exposure alters microtubule motor distribution in adult rat testis. Fundam Appl Tox- icol 1995;24:173–82.

[19] Sibayan SAB, Latina MA, Sherwood ME, Flotte TJ, White K. Apop- tosis and morphology changes in drug-treated trabecular meshwork cells in vitro. Exp Eye Res 1998;66:521–9.

[20] Samms M, Philp D, Emanus F, Osuji O, Pezzano M, Guyden JC.Lysosomal-mediateddegradationofapoptoticthymo- cytes within thymic nurse cells. Cell Immunol 1999;197:108– 15.

[21] Sitte N, Merker K, Grune T, Zglinicki T. Lipofuscin accumulation in proliferating ﬁbroblasts in vitro: an indicator of oxidative stress. Exp Gerontol 2001;36:475–86.

[22] Majdic G, Sharpe RM, O’Shaughnessy PJ, Saunders PTK. Expres- sion of cytochrome P45017-hydroxylase/C17-20 lysase in the fetal rat testis in reduced by maternal exposure to exogenous estrogens. Endocrinology 1996;137:1063–70.

[23] Laurenzana EM, Balasubramanian G, Weis C, Blaydes B, New- bold RR, Delclos KB. Effect of nonylphenol on serum testos- terone levels and testicular steroidogenic enzyme activity in neona- tal, pubertal, and adult rats. Chem Biol Interact 2002;139:23– 41.

[24] Fan Q, Jiang X, Ding X, Jin T. The effects of nonylphenol on inhibin and androgen binding protein expressions in the testis of rats. Acta Acad Med Zunyi 2002;25:197–201.
# The toxic effects of nonylphenol on the reproductive system of male rats

Xiao D. Han,∗, Zhi G. Tu, Yi Gong, Su N. Shen, Xu Y. Wang, Li N. Kang, Ya Y. Hou, Jian X. Chen

 Immunology and Reproductive Biology Laboratory, Medical School, Nanjing University, Nanjing, Jiangsu 210093, PR China  Department of Biology, School of Life Science, Nanjing University, Nanjing, Jiangsu 210093, PR China Received 20 November 2003; received in revised form 16 April 2004; accepted 25 June 2004 Available online 19 August 2004



### Abstract

Male Sprague–Dawley (SD) rats were exposed to $^{4-t}$ -nonylphenol (NP) by gavage at dosages of 0, 125 and  $250\,\mathrm{mg/kg/day}$  for 50 days. Organ weights of liver, kidney, testis and epididymis were measured. Sperm number in the head of epididymis was counted. Several hormones including testosterone, luteinizing hormone and follicle stimulating hormone were measured. Testicular sections were observed by light and electron microscopy. Terminal dideoxynucleotidyl transferase dUTP nick end labeling assay was performed to probe the apoptotic cells in seminiferous tubules. When rats were treated with nonylphenol at  $250\,\mathrm{mg/kg/day},$ , the absolute and relative weight of epididymis decreased dramatically, while the relative weights of kidney and liver increased by 14 and $22\%$ , respectively. In addition, the sperm density of the head of epididymis and the testosterone level descended at  $250\,\mathrm{mg/kg/day}$ . The levels of luteinizing hormone and follicle stimulating hormone increased in both nonylphenol treated groups. Pathological changes were detected by microscopy and the transferase dUTP nick end labeling (TUNEL) assay showed that the number of apoptotic cells in testes increased with nonylphenol in a dose-dependent manner.

 $\copyright2004$  Elsevier Inc. All rights reserved.

Keywords: Nonylphonol; Apoptosis; Reproductive system; Testosterone; Luteinizing hormone; Follicle stimulating hormone; TUNEL

## 1. Introduction

Nonylphenol polyethoxylates (NPEs) are non-ionic sur- factants that are widely used as components of detergents, paints, herbicides, insecticides and many other synthetic products. With their wide use, a large amount of NPEs are charged into water. Nonylphenol (NP), a primary ﬁnal biodegradation product of NPEs [1], has been documented to appear in aquatic environment, particularly in sediment. Its content may reach  $3000\;\mathrm{ppb}$  in rivers and lakes [2]. NP showed a great capacity of bioaccumulation in aquatic an- imals, and investigators found that the NP concentration in the muscle of Anas boscas (living in contaminated aquatic environment) reached  $1.2\,\mathrm{mg/kg}$  [3].

NP has been reported to be weakly estrogenic from in vitro and in vivo studies. In Chapin’s study [4], 3.5 generations of rats were exposed to NP in the diet at 200, 650 and $2000\,\mathrm{ppm}$ , which corresponded to dietary intakes of 9–35, 30–100, and  $100{-}350\,\mathrm{mg/kg/day}$ . Vaginal opening was accelerated by 2 days (at  $650\,\mathrm{ppm})$  and 6 days (at $2000\,\mathrm{ppm})$ ). Uterine weights at postnatal day 21 were increased in F1 females treated with 650 and 2000 ppm NP. In F2 rats, epididymal sperm density was reduced by 8 and  $13\%$  at 650 and 2000 ppm, respec- tively. Collectively, these results suggest that NP has weak estrogenic activity and may induce abnormities in female and male reproductive systems. In Hossaini’s study [5], female pregnant rats were dosed with NP at 3, 15, or  $75\,\mathrm{mg/kg/c}$ day from gestation days 11 to 18. Male offsprings were sacri- ﬁced at postnatal days 11, 21, and 110, and their reproduc- tive parameters were evaluated. The absolute weight of the right epididymis signiﬁcantly decreased in the animals whose mother was exposed to 75 or 15 mg/kg/day NP. In another study,  $100\,\mathrm{mg/kg/day}\,\mathrm{N}$ P reduced the daily sperm production of male rats, and  $300\,\mathrm{mg/kg/day}\ \mathrm{N}$ P induced the atrophy of testis and epididymis [6]. Until now, systemic studies on the male reproductive toxicity of NP have been scarce. The aim of this present study is to further investigate the mechanisms of the reproductive toxicity of NP in male rats.



## 2. Materials and methods

## 2.1. Instruments and reagents

Instruments: Roche Elecsys System (2010), Electron microscope (JEM-1200EX),  $\upgamma$ -counter (SN-695). Sources of reagents were as follows: corn oil (Fortune, Shang- hai, PR China); technical nonylphenol (FGJ 01, purity:  $98\%$ , Tokyo Kasei Kogyo Co. Ltd.); pentobarbital (Sigma, USA); testosterone reagent kit (Roche Diagnostics GmbH, D-68298 Mannheim, Germany); human luteinizing hor- mone radioimmunoassay kit and follicle stimulating hor- mone radioimmunoassay kit (Jiuding Co. Ltd., PR China); apoptosis detection kit (R&D systems, Minneapolis, MN, USA).

## 2.2. Animal treatments

Thirty male Sprague–Dawley rats, aging 28–30 days, were purchased from Nanjing Medical University and housed in groups at  $24\,^{\circ}\mathrm{C}$  in a humidity-controlled room with 12 h light/dark photoperiod. Standard rat chow was ob- tained from Nanjing Medical University (Nanjing Medical University, Nanjing, P.R. China) and available throughout the experiment. Rats were acclimated for 1 week prior to treatments.

Male rats were divided into three groups: control, NP1 and NP2. NP was dissolved in corn oil and given to rats by gavage at  $0\,\mathrm{mg/kg/day}$  (control),  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2). Gavage volume was adjusted accord- ing to the weight of each rat. During the experiment, one rat in the NP2 group manifested a marked reduction in body weight gain and was excluded from the formal experiment. After 50 days treatment, the rats were weighed and anaesthetized by injecting pentobarbital. So far, no report indicates that pen- tobarbital can affect the levels of LH, FSH and testosterone. To obtain pure serum, blood samples were collected via the carotid artery. We inserted a ﬁne tube into the carotid to draw blood into a test tube. When the content of extracted blood reached $3.5–4.5\%$  body weight, this operation was regarded as successful; else it was designated a failure. Because the residual blood in the body will affect organ weight, the cases called failure were excluded from the control, NP1 and NP2 groups as follows: 2, 2 and 1, cases, respectively. At the end of this study, each group had eight rats that can offer available data. After this step, liver, kidney, testis and epididymis were isolated and weighed.

## 2.3. Sperm count

One epididymis of each animal was isolated and cleared of adhering tissues. The head of each epididymis was chopped intosmallpieces(about $1\,\mathrm{mm}^{3}$ )andincubatedin10 mlHam’s F-12 medium at $32\,^{\circ}\mathrm{C}$  for 1 h [7]. Then, 1 ml of the above so- lution was diluted with 9 ml Ham’s F-12 medium. The diluted solution was transferred into each chamber of a haemocy- tometer, and sperm number was counted. To minimize sub- jective error, all operations were performed by one person blinded to the group assignment of animals.

## 2.4. Serum hormone analysis

Serum testosterone concentration was measured by the Roche Elecsys System. The concentrations of LH and FSH were examined by radioimmunoassay, according to the man- ufacture’s instructions. Because commercially available kits are for human serum, the absolute levels of testosterone, FSH and LH could not be qualiﬁed but the comparative levels of these hormone could be ascertained.

## 2.5. Pathological observation

Testicular tissue blocks were ﬁxed in Zenker’s Fluid [8]. Then, samples were sectioned by routine methods followed by H&E staining. Sections were observed by an optical mi- croscope.

## 2.6. TUNEL assay

Testicular tissue blocks were ﬁxed in  $3.7\%$  formaldehyde solution, embedded in parafﬁn and sectioned at $5\,\upmu\mathrm{m}$  for his- tological demonstration of apoptotic fragmentation by the terminal dideoxynucleotidyl transferase dUTP nick end la- beling (TUNEL) method [9,10]. TUNEL-positive cells were counted in 20 randomly selected seminiferous tubules  $(n=$  20).

## 2.7. Cell ultrastructure

Testicular tissue blocks were cut into small pieces, and then ﬁxed in  $2.5\%$  glutaraldehyde. Ultrathin sections were observed by a JEM-1200EX electron microscope.

## 2.8. Statistical analyses

All calculations and statistical analyses were generated in SPSS for windows version 10.0 (SPSS Inc., Chicago, IL, USA). Statistical analyses were performed using one-way analysis of variance (ANOVA) followed by Dunnett’s  $t\mathrm{.}$ -test. Differences were considered to be signiﬁcant at  $P\,<\,0.05$  against control group. Data were presented as mean stan- $\pm$  dard deviation (S.D.).

## 3. Results

There was no signiﬁcant difference in body weight among groups (Table 1). The body weight gain of NP2 group is less than that of the control group, but there is no signiﬁ- cant difference between the two groups. Among rats treated with nonylphenol at  $250\,\mathrm{mg/kg/day}$ , the absolute weight of epididymis decreased dramatically (Table 2). Relative organ weight represents the ratio of organ weight to overall body weight (Table 3). The relative kidney weight of NP2 group increased signiﬁcantly. The relative liver weight increased signiﬁcantly in both NP treated groups. There was no signif- icant difference in relative testis weights among the groups. The relative epididymis weight of the NP2 group dramati- cally decreased. Sperm number in the head of epididymis of the NP2 group dramatically decreased (Table 4).

Serum hormone levels (testosterone, FSH, LH) were mon- itored in male rats and rats exposed to NP (Figs. 1–3). The level of testosterone signiﬁcantly dropped in the NP2 group (Fig. 1). In contrast, FSH (Fig. 2) and LH (Fig. 3) levels in serum were higher in both NP treatment groups versus the control group.

Histopathological examination revealed alterations in the testes of NP1 and NP2 groups (Figs. 4–6). Control animals showed a compact and regular arrangement of cells in the

## 4. Discussion

Nonylphenol, an environmental contaminant, has been reported to induce reproductive abnormalities in male rats [4–6]. In the present study, male SD rats were exposed to  $4–t$ -nonylphenol by gavage at  $125\,\mathrm{mg/kg/day}$  (NP1) and  $250\,\mathrm{mg/kg/day}$  (NP2) for 50 days.

A previous study demonstrated that NP was a low-toxic compound, and the $\mathrm{LD}_{50}$  of NP for rat was  $1475\,\mathrm{mg/kg}$  [11]. Several previous researches [4,5] proved that oral exposure of NP has no adverse inﬂuence on the body weight of rats. In our study, the body weight gain of NP2 group is less than that of the control group, but there was no signiﬁcant difference be- tween the two groups. On the other hand, the increases of the relative weights of kidney and liver in NP groups suggested that NP had chronic toxicity to liver and kidney. Chapin’s study also reported that low-dose NP treatment could pose a hazard to the kidney, while no discernable adverse effect for reproductive system were observed [4].

Our study showed that  $250\,\mathrm{mg/kg/day}\,\mathrm{~N~}$ NP induced the descent of the absolute and relative epididymis weight, and decreased the sperm density in the head of epididymis. These results are consistent with several previous studies.

For example, in de Jager’s study, the total cauda epididymal sperm count was signiﬁcantly lower in the  $250\,\mathrm{mg/kg}$  NP dosage group compared to the control group [12]. In another study [5], the absolute weight of the right epididymis signiﬁcantly decreased in rats exposed to 75 or  $15\,\mathrm{mg/kg/day}$  NP. As for the testis, our result is also consistent with many other reports [4,5,13] that all clariﬁed that NP had no effect on the weight of testis.

In mammals, seminiferous epithelium is composed of two basic cell types: spermatogenic cells and Sertoli cells. In our previous study, NP also caused apoptosis of testicular Sertoli cells in vitro [14]. Apoptosis of germ cells is exaggerated after testicular insults and toxicant exposures [15]. In the present study, TUNEL conﬁrmed that NP treatment increased apoptosis of testicular cells (including Sertoli cells and germ cells). It has long been believed that a Sertoli cell will support a number of developing germ cells and this number is quite constant within a species [16]. So, the dysfunction of Sertoli cells and the apoptosis of Sertoli and germ cells may induce the reduction of the sperm production.

Consistent to the result of TUNEL assay, histological changes were observed in the NP groups. The arrangement of cells in the NP groups was irregular and disordered. Inter- cellular connections were incompact. Distinct ultrastructural changes supported the histopathological assessment. Vac- uolization, for example, is common in testicular injury in the interstitial tissue cells and Sertoli cell where apoptosis was induced with deltamethrin [17]. 2,5-Hexanedione exposure also induced an increase in the number and size of Sertoli cell vacuoles [18]. In our experiment, cellular vacuoles increased dramatically in NP1 and NP2 group. Lysosomes play an im- portant role in cell apoptosis during phagocytosis and au- tophagy. Several previous studies have shown that lysosomes increased and clustered during induced apoptosis [19,20]. A similar increase and inﬂation of lysosome were observed here with NP. In addition, we detected an increase of lipofuscin in testicular cells. Lipofuscin accumulation could be regarded as an indicator of oxidative stress [21], and Chitra’s study showed that lipid peroxidation increased signiﬁcantly in the animals treated with nonylphenol [7]. Hence, the lipofuscin accumulation observed here could be attributed to oxidative stress induced by NP treatment.



It has been reported that NP showed an inhibitory effect on P450c17, an important enzyme in testosterone synthesis [22,23].SothedecreaseoftestosteronelevelintheNP2group might result from the inhibition of P450c17. Previous studies demonstrated that secretion of FSH was regulated by inhibin and testosterone through a negative feedback mechanism. In- hibin production of Sertoli cells was inhibited when rats were exposed to NP [24]. So, we deduce that NP ﬁrstly inhibited the inhibin secretion of Sertoli cells and the testosterone se- cretion of Leydig cells, then the decrease of the inhibin and testosterone secretion resulted in the increase of FSH secre- tion. As the secretion of LH was regulated by testosterone in a negative feedback manner, we deduced that down-regulated testosterone levels in NP groups resulted in the excess se- cretion of LH. In Nagao’s study, 8-week-old male rats were treated with $250\,\mathrm{mg/kg/day\,NP}$  for 12 weeks [13]. Decreases of testosterone level and increases of FSH level were ob- served, but the LH level remained unchanged, which was different from our present study.



Thus, the present data show that relative high dose NP can exert a generally chronic toxicity to rats and a particular toxicity to epididymis, induce the apoptosis of testicular cells (including Sertoli cells and germ cells), and indirectly disturb thesecretionsoftestosterone,LHandFSHandtheproduction of sperm.

## References

[1] Giger W, Brunner PH, Schaffner C. 4-Nonylphenol in sewage sludge: accumulation of toxic metabolites from non-ionic surfactants. Sci- ence 1984;225:623–5.

[2] Naylor CG, Mieure JP, Adams WJ, Weeks JA, Castaldi FJ, Ogle LD, et al. Alkylphenol ethoxylates in the environment. J Am Oil Chem Soc 1992;69:695–703.

[3] Ahel M, Mcevoy J, Giger W. Bioaccumulation of the lipophilic metabolites of nonionic surfactants in freshwater organisms. Env- iron Pollut 1993;79:243–8.

[4] Chapin RE, Delaney J, Wang Y, Lanning L, Davis B, Collins B, et al. The effect of 4-nonylphenol in rats: a multigeneration reproduction study. Toxicol Sci 1999;52:80–91.

[5] Hossaini A, Dalgaard M, Vinggaard AM, Frandsen H, Larsen J. In utero reproductive study in rats exposed to nonylphenol. Reprod Toxicol 2001;15:537–43.

[6] Fan Q, Li W, Shen L, Jiang X, Din X. The effects of nonylphenol on the reproductive functions of male SD rats. J Health Toxicol 2001;15:225–6.

[7] Chitra KC, Latchoumycandane C, Mathur PP. Effect of nonylphenol on the antioxidant system in epididymal sperm of rats. Arch Toxicol 2002;76:545–51.

[8] Zhen GC, editor. Microtechnique in biology. Peking, China: People’s Education Press; 1978. p. 7.

[9] Gavrieli Y, Sherman Y, Ben-Sasson SA. Identiﬁcation of pro- grammed cell death in situ via speciﬁc labeling of nuclear DNA fragmentation. J Cell Biol 1992;119:493–501.

[10] Tilly JL, Perez GL. Mechanisms and genes of physiological cell death: a new direction for toxicological risk assessments? Oxford, UK: Elsevier Press; 1997.

[11] de Jager C, Bornman MS, Wandrag S, Sharp VW. Lethal dose and reproductive parameters of  $p$ -nonylphenol in rats. Arch Androl 2001;46:183–7.

[12] de Jager C, Bornman MS, Oosthuizen JM. The effect of  $p$ - nonylphenol on the fertility potential of male rats after gestational, lactational and direct exposure. Andrologia 1999;31:107–13.

[13] Nagao T, Wadaa K, Marumob H, Yoshimurab S, Onoa H. Repro- ductive effects of nonylphenol in rats after gavage administration: a two-generation study. Reprod Toxicol 2001;15:293–315.

[14] Wang X, Han X, Hou Y, Yao G, Wang Y. Effect of nonylphe- nol on apoptosis of Sertoli cells in vitro. Environ Contam Toxicol 2003;70:898–904.

[15] Lee J, Richburg JH, Shipp EB, Meistrich ML, Boekelheide Kim. The Fas system, a regulator of testicular germ cell apoptosis, is differentially up-regulated in Sertoli cell versus germ cell injury of the testis. Endocrinology 1999;140:852–8.

[16] Sharp RM. Regulation of spermatogenesis. In: Knobl E, Neil JD, editors. The physiology of reproduction. New York: Raven Press; p. 1363–434.

[17] El-Gohary M, Awara WM, Nassar S, Hawas S. Deltamethrin-induced testicular apoptosis in rats: the protective effect of nitric oxide syn- thase inhibitor. Toxicology 1999;132:1–8.

[18] Hall ES, Hall SJ, Boekelheide K. 2,5-Hexanedione exposure alters microtubule motor distribution in adult rat testis. Fundam Appl Tox- icol 1995;24:173–82.

[19] Sibayan SAB, Latina MA, Sherwood ME, Flotte TJ, White K. Apop- tosis and morphology changes in drug-treated trabecular meshwork cells in vitro. Exp Eye Res 1998;66:521–9.

[20] Samms M, Philp D, Emanus F, Osuji O, Pezzano M, Guyden JC.Lysosomal-mediateddegradationofapoptoticthymo- cytes within thymic nurse cells. Cell Immunol 1999;197:108– 15.

[21] Sitte N, Merker K, Grune T, Zglinicki T. Lipofuscin accumulation in proliferating ﬁbroblasts in vitro: an indicator of oxidative stress. Exp Gerontol 2001;36:475–86.

[22] Majdic G, Sharpe RM, O’Shaughnessy PJ, Saunders PTK. Expres- sion of cytochrome P45017-hydroxylase/C17-20 lysase in the fetal rat testis in reduced by maternal exposure to exogenous estrogens. Endocrinology 1996;137:1063–70.

[23] Laurenzana EM, Balasubramanian G, Weis C, Blaydes B, New- bold RR, Delclos KB. Effect of nonylphenol on serum testos- terone levels and testicular steroidogenic enzyme activity in neona- tal, pubertal, and adult rats. Chem Biol Interact 2002;139:23– 41.

[24] Fan Q, Jiang X, Ding X, Jin T. The effects of nonylphenol on inhibin and androgen binding protein expressions in the testis of rats. Acta Acad Med Zunyi 2002;25:197–201.
# Usefulness of the Adenosine Triphosphate with a Sufﬁcient Observation Period for Detecting Reconduction after Pulmonary Vein Isolation

YUICHI NINOMIYA, M.D., YASUHISA IRIKI, M.D., SANEMASA ISHIDA, M.D., NAOYA OKETANI, M.D., TAKEHIKO MATSUSHITA, M.D., HITOSHI ICHIKI, M.D., HIDEKI OKUI, M.D., TETSURO KATAOKA, M.D., AKIKO YOSHIKAWA, M.D., MASAKAZU OGAWA, M.D., KEISHI SAIHARA, M.D., TAKURO SHINSATO, M.D., MASAAKI MIYATA, M.D., SHUICHI HAMASAKI, M.D., and CHUWA TEI, M.D. From the Department of Cardiovascular, Respiratory and Metabolic Medicine, Graduate School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan Background: Although reconduction after pulmonary vein (PV) isolation is considered to play a key role in the recurrence of paroxysmal atrial ﬁbrillation  $(A F)$ , there have been few reports regarding the precise time course of early reconduction. Several studies have suggested that transient PV reconduction facilitated by adenosine may predict long-term  $A F$  recurrence. This study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods: $56\pm\,11$  ential PV antrum ablation was performed in  $a l l\;4\;P V s$ . After the completion of isolation, electrograms in each PV were repeatedly recorded ( $\left(1.98\pm0.57\right.$  times per PV) using a circular mapping catheter for an observation period of  $\ {87\pm29}$  was administered during isoproterenol infusion.

Results: PV electrical isolation was initially achieved in all 81 PVs. During the observation period, 12  $(15\%)$  $\left(15\%\right)P V s\;i n\ 10\left(48\%\right)$  of 21 patients exhibited spontaneous reconduction. Among the remaining 69 PVs, 8  $(12\%)$  additional PVs had reconduction with the use of ATP. All PV reconduction was successfully eliminated by  $4.5\pm2.2$  additional radiofrequency applications.

Conclusion: A sufﬁcient observation period and the use of ATP are useful to detect early reconduction after PV isolation. (PACE 2009; 32:1307–1312)

## adenosine triphosphate, atrial ﬁbrillation, observation period, pulmonary vein isolation, reconduction

Introduction Although reconduction of pulmonary veins (PVs) after electrical PV isolation (PVI) is consid- ered to play a key role in the recurrence of atrial ﬁbrillation (AF), there have been few reports regarding the precise time course of the reconduc- tion during the procedure. Arentz et al. recently re- ported that transient PV reconduction, which may predict long-term AF recurrence, could be facil- itated by adenosine. Matsuo et al. subsequently suggested that additional radiofrequency (RF) en- ergy application to eliminate transient PV recon- nection induced by adenosine triphosphate (ATP) reduced AF recurrence after PVI. The present study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods Study Population The study population consisted of 21 patients  years) with drug re- $56\pm11$  fractory symptomatic AF (paroxysmal, 14; persis- tent, 7 patients). Written informed consent was ob- tained from all patients. All antiarrhythmic drugs were discontinued for at least 5 half-lives before the study.

Preprocedure Measurements The left atrial (LA) dimension, volume, vol- ume index, and left ventricular ejection frac- tion (LVEF) were evaluated by transthoracic echocardiography.PVostiumdiametersand circumferences were measured on multidetector computed tomography (MDCT). The PV ostium was deﬁned as the point of inﬂection between the LA wall and the PV wall. The echocardiography and MDCT evaluators were blinded to the patient’s history. The LA dimension, LA volume, LA vol-  $68\pm18\,\mathrm{ml}$  $40\pm5\,\mathrm{mm}$   $39\pm9\;\mathrm{mL}/\mathrm{m}^{2}$  $67\pm9\%$  Electrophysiologic Study A7-Fhexapolarcatheterwasinserted transnasally into the esophagus. A 14-pole, 2- site mapping lumen catheter (Irvine Biomedical, Inc., Irvine, CA, USA) was positioned in the coro- nary sinus (CS) via the right jugular vein. The transseptal procedure was performed with in- tracardiac echocardiography guidance. Thereafter, intravenous heparin was administered to main- tain the activated clotting time between 250 and 300 seconds. Catheterization into the LA was per- formed using a 1-puncture and 2-sheath tech- nique. Two long vascular sheaths (SL0, St. Jude Medical, Minnetonka, MN, USA) were introduced into the LA. Simultaneous angiography of the ip- silateral 2 PVs using the 6-F Goodtec catheter (Goodman Co., Ltd, Nagoya, Japan) was performed to identify the PV ostia and antrum. PV map- ping was performed with a steerable 20-electrode catheter (OPTIMATM, Irvine Biomedical, Inc). If the catheter could not be deployed in the PVs, a 20-electrode, 15-mm-diameter, circular catheter (Lasso, Biosense Webster, Diamond Bar, CA, USA) was used for mapping those PVs. The catheter size was selected based on the predetermined mea- surement of the PV ostial diameter on angiogra- phy. Electrograms were recorded by the circular mapping catheter during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). When AF persisted during the electrophysiologic study, internal cardioversion using the CS catheter was performed to restore sinus rhythm, and the record- ing during appropriate rhythm was obtained for at least one beat.

  

 



PV Isolation The left superior PV (LSPV), left inferior PV (LIPV), right superior PV (RSPV), and right infe- rior PV (RIPV) were all targeted for isolation. Cir- cumferential PV antrum ablation targeting the PV antrum potentials was performed as previously described. RF energy (CABL-IT II, Japan Lifeline Co., Ltd, Tokyo, Japan) was delivered with a tar- get temperature of  $55^{\circ}\mathrm{C}$  and a maximum power output of  $30~\mathrm{~W~}$  for the LA posterior wall and 35 W elsewhere for 30 seconds using an 8-mm- tip catheter (Ablaze Fantasista, Japan Lifeline Co., Ltd, Tokyo, Japan). If a residual conduction gap was detected after the circumferential ablation, ad- ditional RF applications to the PV side next to the previous RF lesions were delivered. The endpoint of this technique was the completion of PV elec- trical disconnection.



Assessment of PV Electrical Isolation After isolation of each PV, electrograms in the PV were repeatedly recorded using the cir- cular mapping catheter. When reconduction was observed, additional ablation was performed un- til the PV was electrically isolated. After an ob- servation period of 30 minutes or more after the isolation of all 4 PVs, two consecutive series of ATP injections were performed to evaluate dor- mant PV conduction for the right and then the left ipsilateral 2 PVs. The presence/absence of PV conduction was evaluated using two circular map- ping catheters, which were positioned at the right and then the left ipsilateral PVs while the ma- neuver was performed. In all patients, ATP was rapidly injected under isoproterenol (ISP; 0.01–  $0.{\bar{0}}3\,\mathrm{mg/kg/min})$  infusion during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). The ap- pearance of PV conduction was deﬁned as the tran- sient or continuous re-appearance of PV potentials associated with atrial depolarization (Fig. 1). If PV conduction re-appeared, additional RF energy was applied at the earliest PV activation site identiﬁed on the circular mapping catheter until complete PV disconnection was obtained. Complete PV dis- connection was then conﬁrmed by repeat ATP in- jections under ISP infusion.

## Patient Follow-Up

After the PVI procedure, all patients received periodic follow-up (at least once a month) in an outpatient clinic. The presence/absence of AF was evaluated by their symptoms, ECG recordings, and 24-hour ambulatory monitoring (1, 3, 6, and 12 months after the procedure). All patients who had symptoms without documentation of AF re- currence were given an event recorder (available for 5 days) to identify the cause of their symptoms. Freedom from AF was deﬁned as no AF with the patient off medication.

## Statistical Analysis

All values are expressed as the mean  $\pm$  standard deviation (SD). Statistical analysis was conducted using Mann-Whitney’s U test or  $\chi^{2}$  analysis. Differences with  $\mathrm{~P~}<\,\dot{0}.05$  were consid- ered to be statistically signiﬁcant.

Results A common left PV trunk that was observed in three patients was regarded as the LSPV in this study. Therefore, PVI was performed in 21 LSPVs, 18 LIPVs, 21 RSPVs, and 21 RIPVs in all 21 patients. After the completion of PVI, electro- grams in each PV were repeatedly recorded  $(1.98\pm$  0.57 times per PV) using the circular mapping  $\left(25\,\%\right)$ utes). $87\pm29$  PV reconduction was observed in 20 (25%) of 81 PVs in 13  $\left(62\%\right)$  patients after  $66\,\pm\,55$  minutes. Among these 13 patients, one patient had recon- duction in 3 PVs, ﬁve patients had reconduction in 2 PVs, and seven patients had reconduction in 1 PV. The numbers of reconductions observed in the LSPV, LIPV, RSPV, and RIPV were 6, 6, 5, and 3, respectively. Reconduction after an obser- vation period of 60 minutes or more was observed in 2  $\left(10\%\right)$  PVs. PV reconduction was successfully eliminated by additional RF applications  $(4.5\ \pm$  2.2 times) in all patients.

Appearance and Elimination of Dormant PV Conduction In 81 PVs isolated, 12 PVs exhibited spon- taneous reconduction within an observation pe- riod of  $66\,\pm\,55$  minutes. Among the remaining 69 PVs, 8  $\left(12\%\right)$ ) PVs had reconduction with the ATP injection (Fig. 2). Therefore,  $40\%$  of all recon- duction (8 of 20 PVs) was detected only with the use of ATP. All dormant PV reconductions were successfully eliminated by  $3.8\ \pm\ 1.2$  additional RF applications. After additional RF applications, complete PV disconnection was then conﬁrmed by repeat ATP injections under ISP infusion. AF was induced in three patients even after the comple- tion of PVI was conﬁrmed with the ATP injection.

These three patients (paroxysmal, two patients; persistent, one patient) were free from AF after the ablation procedure during a follow-up period of more than one year.

Patient Characteristics with and Without PV Reconduction A higher incidence of reconduction was ob- served in men (13/18) than in women (0/3) ( $\mathrm{~P~}<$  0.05). There was no difference in age, LA volume, LA volume index, or paroxysmal/permanent ratio of AF between groups with and without reconduc- tion (Table I).

 ${\mathsf{N S}}=$ s are mean ± SD. NS = not signiﬁcant.

## PV Characteristics with and Without PV Reconduction

RF energy delivered for the completion of PVI did not signiﬁcantly differ between PVs with sub- sequent rec $15811\,\pm\,8037$  those without ( $,16332\pm$  7243 J vs. 15811 ± 8037 J).  $22\pm5$ was no d $22\pm$   $5\,\mathrm{\mm})$ n PV ostium diamet $(63\,\pm\,13\,\mathrm{{\mm}}$  vs. $65~\pm$   $12\;\mathrm{mm})$  or circumference (63 ± 13 mm vs. 65 ± 12 mm) between PVs with and without reconduc- tion (Table II).

Discussion Major Findings This study demonstrated that a speciﬁc obser- vation period and the use of ATP were essential to detect early reconduction after PVI for AF dur- ing the procedure. This report shows that mean follow-up time was longer than 60 minutes and electrical activation of the PVs could be transiently reactivated only by ATP in  $40\%$  of PVs with recon- duction after successful PV antrum isolation with the interval of at least 30 minutes.

Cheema et al. showed the incidence and time course of PV reconduction after catheter ablation of AF during the procedure. They reported that PV reconduction was observed in 13  $\left(93\%\right)$  pa- tients and 26  $\left(50\%\right)$  veins after complete isola- tion of all PVs. The results revealed an extremely high rate of PV reconduction during the proce- dure. In the present study, the incidence of PV reconduction was less than in their report. The reason may be that we produced a continuous RF lesion using higher RF energy and a longer time. A higher RF energy and longer time applica- tion may increase the risk of fatal complications such as atrioesophageal ﬁstula. Therefore, we used esophageal monitoring to avoid esophageal damage.



Two previous reports described the preva- lence of acute PV reconduction during 60 minutes of observation. In the present study, reconduc- tion was detected over 60 minutes in two  $\left(10\%\right)$  veins. Thus, observation for over 60 minutes may be needed to detect reconduction after PVI during the procedure.

Mechanisms of ATP-Induced Reconduction In a previous report, RF application around the PVs was shown to produce local edema. Lo- cal edema may mask the gap of incomplete PVI due to nontransmural RF lesions, which may re- sult in the a false endpoint of PVI. Since some time is needed for improvement of local edema, the use of ATP may help detect incomplete RF lesions. The mechanism of ATP-induced transient PV re- conduction is not fully understood, but adenosine activates the outward potassium current via the purinergic  $\mathrm{A}_{1}$ -receptor, and the resulting hy- perpolarization of the cell membrane and shorten- ing of the action potential/refractory period may facilitate electronic conduction. Further investi- gation is necessary to clarify the mechanism of ATP induced reconduction.

Inducibility of Atrial Fibrillation Using Adenosine Triphosphate ATP sometimes induces AF, as previously re- ported. In the present study, AF was induced in three patients after PVI with the injection of ATP. Atienza et al. reported the effects of adenosine in- jection on the local activation frequency at differ- ent sites of the ﬁbrillating atria of patients before the ablation. In paroxysmal AF patients, adeno- sine increased activation frequency in the PV–LA junction. In persistent AF patients, the highest- frequency sources accelerated by adenosine were located in either atria but not at PV sites. The re- sults strongly suggest that AF is maintained by re-entrant sources, most likely located at the PV– LA junction in paroxysmal AF, whereas non-PV locations are more likely in persistent AF. In the present study, three patients who had AF induced after successful PVI were free from AF after the ablation procedure. Thus, inducibility of AF after PVI with ATP injection may be related to a non-PV trigger, but it may not be related to long-term AF recurrence.



Patient and PV Characteristics A higher incidence of reconduction was ob- served in men than in women  $(\mathrm{P\<\0.05})$ . Pre- vious studies have consistently shown that men have higher sympathetic activity, whereas women have higher parasympathetic activity. Sympa- thetic activity may be related to the incidence of reconduction.



There was no difference in age, LA volume, and volume index, or the paroxysmal/permanent ratio of AF between the recurrence and nonrecur- rence groups. Yamada et al. demonstrated that an increase in the RF energy needed for PVI was the only predictor of reconduction. However, RF en- ergy for initial PVI did not differ between PVs with reconduction and those without. The factors asso- ciated with reconduction may include myocardial wall thickness, the PV ostium diameter and cir- cumference, and difﬁculty in catheter manipula- tion because of anatomical limitations. It is likely that some of these factors were combined in many cases.

## Limitations

This study possesses several limitations. The ﬁrst is the reproducibility of the effects of ATP. Matsuo et al. assessed the reproducibility of the appearance of PV reconduction with ATP admin- istration. Three methods were compared for the

## References

1. Verma A, Kilicaslan F, Pisano E, Marrouche NF, Fanelli R, Brach- mann J, Geunther J, et al. Response of atrial ﬁbrillation to pul- monary vein antrum isolation is directly related to resumption and delay of pulmonary vein conduction. Circulation 2005; 112:627– 635.

2. Cappato R, Negroni S, Pecora D, Bentivegna S, Lupo PP, Carolei A, Esposito C, et al. Prospective assessment of late conduction recur- rence across radiofrequency lesions producing electrical disconnec- tion at the pulmonary vein ostium in patients with atrial ﬁbrillation. Circulation 2003; 108:1599–1604.

3. Gerstenfeld EP, Callans DJ, Dixit S, Zado E, Marchlinski FE. In- cidence and location of focal atrial ﬁbrillation triggers in pa- tients undergoing repeat pulmonary vein isolation: Implications for ablation strategies. J Cardiovasc Electrophysiol 2003; 14:685– 690.

4. Arentz T, Macle L, Kalusche D, Hocini M, Jais P, Shah D, Hais- saguerre M. “Dormant” pulmonary vein conduction revealed by adenosine after ostial radiofrequency catheter ablation. J Cardio- vasc Electrophysiol 2004; 15:1041–1047.

5. Matsuo S, Yamane T, Date T, Inada K, Kanzaki Y, Tokuda M, Shibayama K, et al. Reduction of AF recurrence after pulmonary vein isolation by eliminating ATP-induced transient venous re- conduction. J Cardiovasc Electrophysiol 2007; 18:704–708.

6. Kato R, Lickfett L, Meininger G, Dickfeld T, Wu R, Juang G, Angkeow P, et al. Pulmonary vein anatomy in patients under- going catheter ablation of atrial ﬁbrillation: Lessons learned by use of magnetic resonance imaging. Circulation 2003; 107:2004– 2010.

7. Yamada T, McElderry HT, Epstein AE, Plumb VJ, Kay GN. One- puncture, double-transseptal catheterization manoeuvre in the catheter ablation of atrial ﬁbrillation. Europace 2007; 9:487–489.

8. Yamada T, Okada T, Murakami Y. A novel catheter for simultaneous angiography of ipsilateral pulmonary veins. Europace 2007; 9:62– 63.

9. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama

induction of dormant PV conduction: ATP  $10\;\mathrm{mg}$ , ATP  $20~\mathrm{mg}$ , and ATP  $20~\mathrm{mg}$  during ISP infusion. Among their 10 cases, PV reconduction could be reproducibly induced only by ATP  $20~\mathrm{mg}$  admin- istration during ISP infusion. On the other hand, Hachiya et al. attempted to provoke PV reconduc- tion by injection of intravenous ATP 30 mg during intravenous isoproterenol infusion  $(1{-}3\ \mu\mathrm{g/min})$  after PVI. Thus, we used  $30~\mathrm{\mg}$  ATP during ISP infusion as the routine method to induce PV reconduction.

The second limitation involved the appropri- ate time to use ATP. We are not aware of any evidence that suggests that ATP should be used immediately after PVI or after a speciﬁc obser- vation period. Therefore, further study is needed to determine the appropriate time for the use of ATP.

Third, this study was neither large population nor randomized. Further examination is expected to be conﬁrmed the results in a larger patient col- lective with randomized assignment to the pro- posed technique versus a conventional one.

## Conclusion

The present ﬁndings suggest that a sufﬁcient observation period and the use of ATP are use- ful to detect reconduction after PVI during the procedure.

J, Yoshida Y, et al. Electrophysiologic pulmonary vein antrum iso- lation with a multielectrode basket catheter is feasible and effec- tive for curing paroxysmal atrial ﬁbrillation: Efﬁcacy of minimally extensive pulmonary vein isolation. Heart Rhythm 2006; 4:377– 384.

10. Cheema A, Dong J, Dalal D, Marine JE, Henrikson CA, Spragg D, Cheng A, et al. Incidence and time course of early recovery of pul- monary vein conduction after catheter ablation of atrial ﬁbrillation. J Cardiovasc Electrophysiol 2007; 18:387–391.

11. Pappone C, Oral H, Santinelli V, Vicedomini G, Lang CC, Manguso F, Torracca L, et al. Atrio-esophageal ﬁstula as a complication of percutaneous transcatheter ablation of atrial ﬁbrillation. Circulation 2004; 109:2724–2726.

12. Wang XH, Liu X, Sun YM, Gu JN, Shi HF, Zhou L, Hu W. Early identiﬁcation and treatment of PV re-connections: Role of observa- tion time and impact on clinical results of atrial ﬁbrillation ablation. Europace 2007; 9:481–486.

13. Okada T, Yamada T, Murakami Y, Yoshida N, Ninomiya Y, Shimizu T, Toyama J, et al. Prevalence and severity of left atrial edema de- tected by electron beam tomography early after pulmonary vein ablation. J Am Coll Cardiol 2007; 49:1436–1442.

14. Berlardinelli L, Linden J, Berne RM. The cardiac effects of adeno- sine. Prog Cardiovasc Dis 1989; 32:73–97.

15. Lerman B, Berlardinelli L. Cardiac electrophysiology of adenosine. Basic and clinical concepts. Circulation 1991; 83:1499–1509.

16. Belhassen B, Pelleg A, Shoshani D, Laniado S. Atrial ﬁbrillation induced by adenosine triphosphate. Am J Cardiol 1984; 53:1405– 1406.

17. Atienza F, Almendral J, Moreno J, Vaidyanathan R, Talkachou A, Kalifa J, Arenal A, et al. Activation of inward rectiﬁer potassium channels accelerates atrial ﬁbrillation in humans: Evidence for a reentrant mechanism. Circulation 2006; 114:2434–2442.

18. Jones PP, Snitker S, Skinner JS, Ravussin E. Gender differences in muscle sympathetic nerve activity: Effect of body fat distribution. Am J Physiol 1996; 270:E363–6.

19. Yamasaki Y, Kodama M, Matsuhisa M, Kishimoto M, Ozaki H, Tani A, Ueda N, et al. Diurnal heart rate variability in healthy sub- jects: Effects of aging and sex difference. Am J Physiol 1996; 271: H303–10.

20. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama J, Yoshida Y, et al. Incidence, location, and cause of recovery of electrical connections between the pulmonary veins and the left atrium after pulmonary vein isolation. Europace 2006; 8:182– 188.



21. Hachiya H, Hirao K, Takahashi A, Nagata Y, Suzuki K, Maeda S, Sasaki T, et al. Clinical implications of reconnection between the left atrium and isolated pulmonary veins provoked by adenosine triphosphate after extensive encircling pulmonary vein isolation. J Cardiovasc Electrophysiol 2007; 18:392–398.
# Usefulness of the Adenosine Triphosphate with a Sufﬁcient Observation Period for Detecting Reconduction after Pulmonary Vein Isolation

YUICHI NINOMIYA, M.D., YASUHISA IRIKI, M.D., SANEMASA ISHIDA, M.D., NAOYA OKETANI, M.D., TAKEHIKO MATSUSHITA, M.D., HITOSHI ICHIKI, M.D., HIDEKI OKUI, M.D., TETSURO KATAOKA, M.D., AKIKO YOSHIKAWA, M.D., MASAKAZU OGAWA, M.D., KEISHI SAIHARA, M.D., TAKURO SHINSATO, M.D., MASAAKI MIYATA, M.D., SHUICHI HAMASAKI, M.D., and CHUWA TEI, M.D. From the Department of Cardiovascular, Respiratory and Metabolic Medicine, Graduate School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan Background: Although reconduction after pulmonary vein (PV) isolation is considered to play a key role in the recurrence of paroxysmal atrial ﬁbrillation  $(A F)$ , there have been few reports regarding the precise time course of early reconduction. Several studies have suggested that transient PV reconduction facilitated by adenosine may predict long-term  $A F$  recurrence. This study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods: $56\pm\,11$  ential PV antrum ablation was performed in  $a l l\;4\;P V s$ . After the completion of isolation, electrograms in each PV were repeatedly recorded ( $\left(1.98\pm0.57\right.$  times per PV) using a circular mapping catheter for an observation period of  $\ {87\pm29}$  was administered during isoproterenol infusion.

Results: PV electrical isolation was initially achieved in all 81 PVs. During the observation period, 12  $(15\%)$  $\left(15\%\right)P V s\;i n\ 10\left(48\%\right)$  of 21 patients exhibited spontaneous reconduction. Among the remaining 69 PVs, 8  $(12\%)$  additional PVs had reconduction with the use of ATP. All PV reconduction was successfully eliminated by  $4.5\pm2.2$  additional radiofrequency applications.

Conclusion: A sufﬁcient observation period and the use of ATP are useful to detect early reconduction after PV isolation. (PACE 2009; 32:1307–1312)

## adenosine triphosphate, atrial ﬁbrillation, observation period, pulmonary vein isolation, reconduction

Introduction Although reconduction of pulmonary veins (PVs) after electrical PV isolation (PVI) is consid- ered to play a key role in the recurrence of atrial ﬁbrillation (AF), there have been few reports regarding the precise time course of the reconduc- tion during the procedure. Arentz et al. recently re- ported that transient PV reconduction, which may predict long-term AF recurrence, could be facil- itated by adenosine. Matsuo et al. subsequently suggested that additional radiofrequency (RF) en- ergy application to eliminate transient PV recon- nection induced by adenosine triphosphate (ATP) reduced AF recurrence after PVI. The present study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods Study Population The study population consisted of 21 patients  years) with drug re- $56\pm11$  fractory symptomatic AF (paroxysmal, 14; persis- tent, 7 patients). Written informed consent was ob- tained from all patients. All antiarrhythmic drugs were discontinued for at least 5 half-lives before the study.

Preprocedure Measurements The left atrial (LA) dimension, volume, vol- ume index, and left ventricular ejection frac- tion (LVEF) were evaluated by transthoracic echocardiography.PVostiumdiametersand circumferences were measured on multidetector computed tomography (MDCT). The PV ostium was deﬁned as the point of inﬂection between the LA wall and the PV wall. The echocardiography and MDCT evaluators were blinded to the patient’s history. The LA dimension, LA volume, LA vol-  $68\pm18\,\mathrm{ml}$  $40\pm5\,\mathrm{mm}$   $39\pm9\;\mathrm{mL}/\mathrm{m}^{2}$  $67\pm9\%$  Electrophysiologic Study A7-Fhexapolarcatheterwasinserted transnasally into the esophagus. A 14-pole, 2- site mapping lumen catheter (Irvine Biomedical, Inc., Irvine, CA, USA) was positioned in the coro- nary sinus (CS) via the right jugular vein. The transseptal procedure was performed with in- tracardiac echocardiography guidance. Thereafter, intravenous heparin was administered to main- tain the activated clotting time between 250 and 300 seconds. Catheterization into the LA was per- formed using a 1-puncture and 2-sheath tech- nique. Two long vascular sheaths (SL0, St. Jude Medical, Minnetonka, MN, USA) were introduced into the LA. Simultaneous angiography of the ip- silateral 2 PVs using the 6-F Goodtec catheter (Goodman Co., Ltd, Nagoya, Japan) was performed to identify the PV ostia and antrum. PV map- ping was performed with a steerable 20-electrode catheter (OPTIMATM, Irvine Biomedical, Inc). If the catheter could not be deployed in the PVs, a 20-electrode, 15-mm-diameter, circular catheter (Lasso, Biosense Webster, Diamond Bar, CA, USA) was used for mapping those PVs. The catheter size was selected based on the predetermined mea- surement of the PV ostial diameter on angiogra- phy. Electrograms were recorded by the circular mapping catheter during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). When AF persisted during the electrophysiologic study, internal cardioversion using the CS catheter was performed to restore sinus rhythm, and the record- ing during appropriate rhythm was obtained for at least one beat.

  

 



PV Isolation The left superior PV (LSPV), left inferior PV (LIPV), right superior PV (RSPV), and right infe- rior PV (RIPV) were all targeted for isolation. Cir- cumferential PV antrum ablation targeting the PV antrum potentials was performed as previously described. RF energy (CABL-IT II, Japan Lifeline Co., Ltd, Tokyo, Japan) was delivered with a tar- get temperature of  $55^{\circ}\mathrm{C}$  and a maximum power output of  $30~\mathrm{~W~}$  for the LA posterior wall and 35 W elsewhere for 30 seconds using an 8-mm- tip catheter (Ablaze Fantasista, Japan Lifeline Co., Ltd, Tokyo, Japan). If a residual conduction gap was detected after the circumferential ablation, ad- ditional RF applications to the PV side next to the previous RF lesions were delivered. The endpoint of this technique was the completion of PV elec- trical disconnection.



Assessment of PV Electrical Isolation After isolation of each PV, electrograms in the PV were repeatedly recorded using the cir- cular mapping catheter. When reconduction was observed, additional ablation was performed un- til the PV was electrically isolated. After an ob- servation period of 30 minutes or more after the isolation of all 4 PVs, two consecutive series of ATP injections were performed to evaluate dor- mant PV conduction for the right and then the left ipsilateral 2 PVs. The presence/absence of PV conduction was evaluated using two circular map- ping catheters, which were positioned at the right and then the left ipsilateral PVs while the ma- neuver was performed. In all patients, ATP was rapidly injected under isoproterenol (ISP; 0.01–  $0.{\bar{0}}3\,\mathrm{mg/kg/min})$  infusion during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). The ap- pearance of PV conduction was deﬁned as the tran- sient or continuous re-appearance of PV potentials associated with atrial depolarization (Fig. 1). If PV conduction re-appeared, additional RF energy was applied at the earliest PV activation site identiﬁed on the circular mapping catheter until complete PV disconnection was obtained. Complete PV dis- connection was then conﬁrmed by repeat ATP in- jections under ISP infusion.

## Patient Follow-Up

After the PVI procedure, all patients received periodic follow-up (at least once a month) in an outpatient clinic. The presence/absence of AF was evaluated by their symptoms, ECG recordings, and 24-hour ambulatory monitoring (1, 3, 6, and 12 months after the procedure). All patients who had symptoms without documentation of AF re- currence were given an event recorder (available for 5 days) to identify the cause of their symptoms. Freedom from AF was deﬁned as no AF with the patient off medication.

## Statistical Analysis

All values are expressed as the mean  $\pm$  standard deviation (SD). Statistical analysis was conducted using Mann-Whitney’s U test or  $\chi^{2}$  analysis. Differences with  $\mathrm{~P~}<\,\dot{0}.05$  were consid- ered to be statistically signiﬁcant.

Results A common left PV trunk that was observed in three patients was regarded as the LSPV in this study. Therefore, PVI was performed in 21 LSPVs, 18 LIPVs, 21 RSPVs, and 21 RIPVs in all 21 patients. After the completion of PVI, electro- grams in each PV were repeatedly recorded  $(1.98\pm$  0.57 times per PV) using the circular mapping  $\left(25\,\%\right)$ utes). $87\pm29$  PV reconduction was observed in 20 (25%) of 81 PVs in 13  $\left(62\%\right)$  patients after  $66\,\pm\,55$  minutes. Among these 13 patients, one patient had recon- duction in 3 PVs, ﬁve patients had reconduction in 2 PVs, and seven patients had reconduction in 1 PV. The numbers of reconductions observed in the LSPV, LIPV, RSPV, and RIPV were 6, 6, 5, and 3, respectively. Reconduction after an obser- vation period of 60 minutes or more was observed in 2  $\left(10\%\right)$  PVs. PV reconduction was successfully eliminated by additional RF applications  $(4.5\ \pm$  2.2 times) in all patients.

Appearance and Elimination of Dormant PV Conduction In 81 PVs isolated, 12 PVs exhibited spon- taneous reconduction within an observation pe- riod of  $66\,\pm\,55$  minutes. Among the remaining 69 PVs, 8  $\left(12\%\right)$ ) PVs had reconduction with the ATP injection (Fig. 2). Therefore,  $40\%$  of all recon- duction (8 of 20 PVs) was detected only with the use of ATP. All dormant PV reconductions were successfully eliminated by  $3.8\ \pm\ 1.2$  additional RF applications. After additional RF applications, complete PV disconnection was then conﬁrmed by repeat ATP injections under ISP infusion. AF was induced in three patients even after the comple- tion of PVI was conﬁrmed with the ATP injection.

These three patients (paroxysmal, two patients; persistent, one patient) were free from AF after the ablation procedure during a follow-up period of more than one year.

Patient Characteristics with and Without PV Reconduction A higher incidence of reconduction was ob- served in men (13/18) than in women (0/3) ( $\mathrm{~P~}<$  0.05). There was no difference in age, LA volume, LA volume index, or paroxysmal/permanent ratio of AF between groups with and without reconduc- tion (Table I).

 ${\mathsf{N S}}=$ s are mean ± SD. NS = not signiﬁcant.

## PV Characteristics with and Without PV Reconduction

RF energy delivered for the completion of PVI did not signiﬁcantly differ between PVs with sub- sequent rec $15811\,\pm\,8037$  those without ( $,16332\pm$  7243 J vs. 15811 ± 8037 J).  $22\pm5$ was no d $22\pm$   $5\,\mathrm{\mm})$ n PV ostium diamet $(63\,\pm\,13\,\mathrm{{\mm}}$  vs. $65~\pm$   $12\;\mathrm{mm})$  or circumference (63 ± 13 mm vs. 65 ± 12 mm) between PVs with and without reconduc- tion (Table II).

Discussion Major Findings This study demonstrated that a speciﬁc obser- vation period and the use of ATP were essential to detect early reconduction after PVI for AF dur- ing the procedure. This report shows that mean follow-up time was longer than 60 minutes and electrical activation of the PVs could be transiently reactivated only by ATP in  $40\%$  of PVs with recon- duction after successful PV antrum isolation with the interval of at least 30 minutes.

Cheema et al. showed the incidence and time course of PV reconduction after catheter ablation of AF during the procedure. They reported that PV reconduction was observed in 13  $\left(93\%\right)$  pa- tients and 26  $\left(50\%\right)$  veins after complete isola- tion of all PVs. The results revealed an extremely high rate of PV reconduction during the proce- dure. In the present study, the incidence of PV reconduction was less than in their report. The reason may be that we produced a continuous RF lesion using higher RF energy and a longer time. A higher RF energy and longer time applica- tion may increase the risk of fatal complications such as atrioesophageal ﬁstula. Therefore, we used esophageal monitoring to avoid esophageal damage.



Two previous reports described the preva- lence of acute PV reconduction during 60 minutes of observation. In the present study, reconduc- tion was detected over 60 minutes in two  $\left(10\%\right)$  veins. Thus, observation for over 60 minutes may be needed to detect reconduction after PVI during the procedure.

Mechanisms of ATP-Induced Reconduction In a previous report, RF application around the PVs was shown to produce local edema. Lo- cal edema may mask the gap of incomplete PVI due to nontransmural RF lesions, which may re- sult in the a false endpoint of PVI. Since some time is needed for improvement of local edema, the use of ATP may help detect incomplete RF lesions. The mechanism of ATP-induced transient PV re- conduction is not fully understood, but adenosine activates the outward potassium current via the purinergic  $\mathrm{A}_{1}$ -receptor, and the resulting hy- perpolarization of the cell membrane and shorten- ing of the action potential/refractory period may facilitate electronic conduction. Further investi- gation is necessary to clarify the mechanism of ATP induced reconduction.

Inducibility of Atrial Fibrillation Using Adenosine Triphosphate ATP sometimes induces AF, as previously re- ported. In the present study, AF was induced in three patients after PVI with the injection of ATP. Atienza et al. reported the effects of adenosine in- jection on the local activation frequency at differ- ent sites of the ﬁbrillating atria of patients before the ablation. In paroxysmal AF patients, adeno- sine increased activation frequency in the PV–LA junction. In persistent AF patients, the highest- frequency sources accelerated by adenosine were located in either atria but not at PV sites. The re- sults strongly suggest that AF is maintained by re-entrant sources, most likely located at the PV– LA junction in paroxysmal AF, whereas non-PV locations are more likely in persistent AF. In the present study, three patients who had AF induced after successful PVI were free from AF after the ablation procedure. Thus, inducibility of AF after PVI with ATP injection may be related to a non-PV trigger, but it may not be related to long-term AF recurrence.



Patient and PV Characteristics A higher incidence of reconduction was ob- served in men than in women  $(\mathrm{P\<\0.05})$ . Pre- vious studies have consistently shown that men have higher sympathetic activity, whereas women have higher parasympathetic activity. Sympa- thetic activity may be related to the incidence of reconduction.



There was no difference in age, LA volume, and volume index, or the paroxysmal/permanent ratio of AF between the recurrence and nonrecur- rence groups. Yamada et al. demonstrated that an increase in the RF energy needed for PVI was the only predictor of reconduction. However, RF en- ergy for initial PVI did not differ between PVs with reconduction and those without. The factors asso- ciated with reconduction may include myocardial wall thickness, the PV ostium diameter and cir- cumference, and difﬁculty in catheter manipula- tion because of anatomical limitations. It is likely that some of these factors were combined in many cases.

## Limitations

This study possesses several limitations. The ﬁrst is the reproducibility of the effects of ATP. Matsuo et al. assessed the reproducibility of the appearance of PV reconduction with ATP admin- istration. Three methods were compared for the

## References

1. Verma A, Kilicaslan F, Pisano E, Marrouche NF, Fanelli R, Brach- mann J, Geunther J, et al. Response of atrial ﬁbrillation to pul- monary vein antrum isolation is directly related to resumption and delay of pulmonary vein conduction. Circulation 2005; 112:627– 635.

2. Cappato R, Negroni S, Pecora D, Bentivegna S, Lupo PP, Carolei A, Esposito C, et al. Prospective assessment of late conduction recur- rence across radiofrequency lesions producing electrical disconnec- tion at the pulmonary vein ostium in patients with atrial ﬁbrillation. Circulation 2003; 108:1599–1604.

3. Gerstenfeld EP, Callans DJ, Dixit S, Zado E, Marchlinski FE. In- cidence and location of focal atrial ﬁbrillation triggers in pa- tients undergoing repeat pulmonary vein isolation: Implications for ablation strategies. J Cardiovasc Electrophysiol 2003; 14:685– 690.

4. Arentz T, Macle L, Kalusche D, Hocini M, Jais P, Shah D, Hais- saguerre M. “Dormant” pulmonary vein conduction revealed by adenosine after ostial radiofrequency catheter ablation. J Cardio- vasc Electrophysiol 2004; 15:1041–1047.

5. Matsuo S, Yamane T, Date T, Inada K, Kanzaki Y, Tokuda M, Shibayama K, et al. Reduction of AF recurrence after pulmonary vein isolation by eliminating ATP-induced transient venous re- conduction. J Cardiovasc Electrophysiol 2007; 18:704–708.

6. Kato R, Lickfett L, Meininger G, Dickfeld T, Wu R, Juang G, Angkeow P, et al. Pulmonary vein anatomy in patients under- going catheter ablation of atrial ﬁbrillation: Lessons learned by use of magnetic resonance imaging. Circulation 2003; 107:2004– 2010.

7. Yamada T, McElderry HT, Epstein AE, Plumb VJ, Kay GN. One- puncture, double-transseptal catheterization manoeuvre in the catheter ablation of atrial ﬁbrillation. Europace 2007; 9:487–489.

8. Yamada T, Okada T, Murakami Y. A novel catheter for simultaneous angiography of ipsilateral pulmonary veins. Europace 2007; 9:62– 63.

9. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama

induction of dormant PV conduction: ATP  $10\;\mathrm{mg}$ , ATP  $20~\mathrm{mg}$ , and ATP  $20~\mathrm{mg}$  during ISP infusion. Among their 10 cases, PV reconduction could be reproducibly induced only by ATP  $20~\mathrm{mg}$  admin- istration during ISP infusion. On the other hand, Hachiya et al. attempted to provoke PV reconduc- tion by injection of intravenous ATP 30 mg during intravenous isoproterenol infusion  $(1{-}3\ \mu\mathrm{g/min})$  after PVI. Thus, we used  $30~\mathrm{\mg}$  ATP during ISP infusion as the routine method to induce PV reconduction.

The second limitation involved the appropri- ate time to use ATP. We are not aware of any evidence that suggests that ATP should be used immediately after PVI or after a speciﬁc obser- vation period. Therefore, further study is needed to determine the appropriate time for the use of ATP.

Third, this study was neither large population nor randomized. Further examination is expected to be conﬁrmed the results in a larger patient col- lective with randomized assignment to the pro- posed technique versus a conventional one.

## Conclusion

The present ﬁndings suggest that a sufﬁcient observation period and the use of ATP are use- ful to detect reconduction after PVI during the procedure.

J, Yoshida Y, et al. Electrophysiologic pulmonary vein antrum iso- lation with a multielectrode basket catheter is feasible and effec- tive for curing paroxysmal atrial ﬁbrillation: Efﬁcacy of minimally extensive pulmonary vein isolation. Heart Rhythm 2006; 4:377– 384.

10. Cheema A, Dong J, Dalal D, Marine JE, Henrikson CA, Spragg D, Cheng A, et al. Incidence and time course of early recovery of pul- monary vein conduction after catheter ablation of atrial ﬁbrillation. J Cardiovasc Electrophysiol 2007; 18:387–391.

11. Pappone C, Oral H, Santinelli V, Vicedomini G, Lang CC, Manguso F, Torracca L, et al. Atrio-esophageal ﬁstula as a complication of percutaneous transcatheter ablation of atrial ﬁbrillation. Circulation 2004; 109:2724–2726.

12. Wang XH, Liu X, Sun YM, Gu JN, Shi HF, Zhou L, Hu W. Early identiﬁcation and treatment of PV re-connections: Role of observa- tion time and impact on clinical results of atrial ﬁbrillation ablation. Europace 2007; 9:481–486.

13. Okada T, Yamada T, Murakami Y, Yoshida N, Ninomiya Y, Shimizu T, Toyama J, et al. Prevalence and severity of left atrial edema de- tected by electron beam tomography early after pulmonary vein ablation. J Am Coll Cardiol 2007; 49:1436–1442.

14. Berlardinelli L, Linden J, Berne RM. The cardiac effects of adeno- sine. Prog Cardiovasc Dis 1989; 32:73–97.

15. Lerman B, Berlardinelli L. Cardiac electrophysiology of adenosine. Basic and clinical concepts. Circulation 1991; 83:1499–1509.

16. Belhassen B, Pelleg A, Shoshani D, Laniado S. Atrial ﬁbrillation induced by adenosine triphosphate. Am J Cardiol 1984; 53:1405– 1406.

17. Atienza F, Almendral J, Moreno J, Vaidyanathan R, Talkachou A, Kalifa J, Arenal A, et al. Activation of inward rectiﬁer potassium channels accelerates atrial ﬁbrillation in humans: Evidence for a reentrant mechanism. Circulation 2006; 114:2434–2442.

18. Jones PP, Snitker S, Skinner JS, Ravussin E. Gender differences in muscle sympathetic nerve activity: Effect of body fat distribution. Am J Physiol 1996; 270:E363–6.

19. Yamasaki Y, Kodama M, Matsuhisa M, Kishimoto M, Ozaki H, Tani A, Ueda N, et al. Diurnal heart rate variability in healthy sub- jects: Effects of aging and sex difference. Am J Physiol 1996; 271: H303–10.

20. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama J, Yoshida Y, et al. Incidence, location, and cause of recovery of electrical connections between the pulmonary veins and the left atrium after pulmonary vein isolation. Europace 2006; 8:182– 188.



21. Hachiya H, Hirao K, Takahashi A, Nagata Y, Suzuki K, Maeda S, Sasaki T, et al. Clinical implications of reconnection between the left atrium and isolated pulmonary veins provoked by adenosine triphosphate after extensive encircling pulmonary vein isolation. J Cardiovasc Electrophysiol 2007; 18:392–398.
# Usefulness of the Adenosine Triphosphate with a Sufﬁcient Observation Period for Detecting Reconduction after Pulmonary Vein Isolation

YUICHI NINOMIYA, M.D., YASUHISA IRIKI, M.D., SANEMASA ISHIDA, M.D., NAOYA OKETANI, M.D., TAKEHIKO MATSUSHITA, M.D., HITOSHI ICHIKI, M.D., HIDEKI OKUI, M.D., TETSURO KATAOKA, M.D., AKIKO YOSHIKAWA, M.D., MASAKAZU OGAWA, M.D., KEISHI SAIHARA, M.D., TAKURO SHINSATO, M.D., MASAAKI MIYATA, M.D., SHUICHI HAMASAKI, M.D., and CHUWA TEI, M.D. From the Department of Cardiovascular, Respiratory and Metabolic Medicine, Graduate School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan Background: Although reconduction after pulmonary vein (PV) isolation is considered to play a key role in the recurrence of paroxysmal atrial ﬁbrillation  $(A F)$ , there have been few reports regarding the precise time course of early reconduction. Several studies have suggested that transient PV reconduction facilitated by adenosine may predict long-term  $A F$  recurrence. This study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods: $56\pm\,11$  ential PV antrum ablation was performed in  $a l l\;4\;P V s$ . After the completion of isolation, electrograms in each PV were repeatedly recorded ( $\left(1.98\pm0.57\right.$  times per PV) using a circular mapping catheter for an observation period of  $\ {87\pm29}$  was administered during isoproterenol infusion.

Results: PV electrical isolation was initially achieved in all 81 PVs. During the observation period, 12  $(15\%)$  $\left(15\%\right)P V s\;i n\ 10\left(48\%\right)$  of 21 patients exhibited spontaneous reconduction. Among the remaining 69 PVs, 8  $(12\%)$  additional PVs had reconduction with the use of ATP. All PV reconduction was successfully eliminated by  $4.5\pm2.2$  additional radiofrequency applications.

Conclusion: A sufﬁcient observation period and the use of ATP are useful to detect early reconduction after PV isolation. (PACE 2009; 32:1307–1312)

## adenosine triphosphate, atrial ﬁbrillation, observation period, pulmonary vein isolation, reconduction

Introduction Although reconduction of pulmonary veins (PVs) after electrical PV isolation (PVI) is consid- ered to play a key role in the recurrence of atrial ﬁbrillation (AF), there have been few reports regarding the precise time course of the reconduc- tion during the procedure. Arentz et al. recently re- ported that transient PV reconduction, which may predict long-term AF recurrence, could be facil- itated by adenosine. Matsuo et al. subsequently suggested that additional radiofrequency (RF) en- ergy application to eliminate transient PV recon- nection induced by adenosine triphosphate (ATP) reduced AF recurrence after PVI. The present study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods Study Population The study population consisted of 21 patients  years) with drug re- $56\pm11$  fractory symptomatic AF (paroxysmal, 14; persis- tent, 7 patients). Written informed consent was ob- tained from all patients. All antiarrhythmic drugs were discontinued for at least 5 half-lives before the study.

Preprocedure Measurements The left atrial (LA) dimension, volume, vol- ume index, and left ventricular ejection frac- tion (LVEF) were evaluated by transthoracic echocardiography.PVostiumdiametersand circumferences were measured on multidetector computed tomography (MDCT). The PV ostium was deﬁned as the point of inﬂection between the LA wall and the PV wall. The echocardiography and MDCT evaluators were blinded to the patient’s history. The LA dimension, LA volume, LA vol-  $68\pm18\,\mathrm{ml}$  $40\pm5\,\mathrm{mm}$   $39\pm9\;\mathrm{mL}/\mathrm{m}^{2}$  $67\pm9\%$  Electrophysiologic Study A7-Fhexapolarcatheterwasinserted transnasally into the esophagus. A 14-pole, 2- site mapping lumen catheter (Irvine Biomedical, Inc., Irvine, CA, USA) was positioned in the coro- nary sinus (CS) via the right jugular vein. The transseptal procedure was performed with in- tracardiac echocardiography guidance. Thereafter, intravenous heparin was administered to main- tain the activated clotting time between 250 and 300 seconds. Catheterization into the LA was per- formed using a 1-puncture and 2-sheath tech- nique. Two long vascular sheaths (SL0, St. Jude Medical, Minnetonka, MN, USA) were introduced into the LA. Simultaneous angiography of the ip- silateral 2 PVs using the 6-F Goodtec catheter (Goodman Co., Ltd, Nagoya, Japan) was performed to identify the PV ostia and antrum. PV map- ping was performed with a steerable 20-electrode catheter (OPTIMATM, Irvine Biomedical, Inc). If the catheter could not be deployed in the PVs, a 20-electrode, 15-mm-diameter, circular catheter (Lasso, Biosense Webster, Diamond Bar, CA, USA) was used for mapping those PVs. The catheter size was selected based on the predetermined mea- surement of the PV ostial diameter on angiogra- phy. Electrograms were recorded by the circular mapping catheter during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). When AF persisted during the electrophysiologic study, internal cardioversion using the CS catheter was performed to restore sinus rhythm, and the record- ing during appropriate rhythm was obtained for at least one beat.

  

 



PV Isolation The left superior PV (LSPV), left inferior PV (LIPV), right superior PV (RSPV), and right infe- rior PV (RIPV) were all targeted for isolation. Cir- cumferential PV antrum ablation targeting the PV antrum potentials was performed as previously described. RF energy (CABL-IT II, Japan Lifeline Co., Ltd, Tokyo, Japan) was delivered with a tar- get temperature of  $55^{\circ}\mathrm{C}$  and a maximum power output of  $30~\mathrm{~W~}$  for the LA posterior wall and 35 W elsewhere for 30 seconds using an 8-mm- tip catheter (Ablaze Fantasista, Japan Lifeline Co., Ltd, Tokyo, Japan). If a residual conduction gap was detected after the circumferential ablation, ad- ditional RF applications to the PV side next to the previous RF lesions were delivered. The endpoint of this technique was the completion of PV elec- trical disconnection.



Assessment of PV Electrical Isolation After isolation of each PV, electrograms in the PV were repeatedly recorded using the cir- cular mapping catheter. When reconduction was observed, additional ablation was performed un- til the PV was electrically isolated. After an ob- servation period of 30 minutes or more after the isolation of all 4 PVs, two consecutive series of ATP injections were performed to evaluate dor- mant PV conduction for the right and then the left ipsilateral 2 PVs. The presence/absence of PV conduction was evaluated using two circular map- ping catheters, which were positioned at the right and then the left ipsilateral PVs while the ma- neuver was performed. In all patients, ATP was rapidly injected under isoproterenol (ISP; 0.01–  $0.{\bar{0}}3\,\mathrm{mg/kg/min})$  infusion during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). The ap- pearance of PV conduction was deﬁned as the tran- sient or continuous re-appearance of PV potentials associated with atrial depolarization (Fig. 1). If PV conduction re-appeared, additional RF energy was applied at the earliest PV activation site identiﬁed on the circular mapping catheter until complete PV disconnection was obtained. Complete PV dis- connection was then conﬁrmed by repeat ATP in- jections under ISP infusion.

## Patient Follow-Up

After the PVI procedure, all patients received periodic follow-up (at least once a month) in an outpatient clinic. The presence/absence of AF was evaluated by their symptoms, ECG recordings, and 24-hour ambulatory monitoring (1, 3, 6, and 12 months after the procedure). All patients who had symptoms without documentation of AF re- currence were given an event recorder (available for 5 days) to identify the cause of their symptoms. Freedom from AF was deﬁned as no AF with the patient off medication.

## Statistical Analysis

All values are expressed as the mean  $\pm$  standard deviation (SD). Statistical analysis was conducted using Mann-Whitney’s U test or  $\chi^{2}$  analysis. Differences with  $\mathrm{~P~}<\,\dot{0}.05$  were consid- ered to be statistically signiﬁcant.

Results A common left PV trunk that was observed in three patients was regarded as the LSPV in this study. Therefore, PVI was performed in 21 LSPVs, 18 LIPVs, 21 RSPVs, and 21 RIPVs in all 21 patients. After the completion of PVI, electro- grams in each PV were repeatedly recorded  $(1.98\pm$  0.57 times per PV) using the circular mapping  $\left(25\,\%\right)$ utes). $87\pm29$  PV reconduction was observed in 20 (25%) of 81 PVs in 13  $\left(62\%\right)$  patients after  $66\,\pm\,55$  minutes. Among these 13 patients, one patient had recon- duction in 3 PVs, ﬁve patients had reconduction in 2 PVs, and seven patients had reconduction in 1 PV. The numbers of reconductions observed in the LSPV, LIPV, RSPV, and RIPV were 6, 6, 5, and 3, respectively. Reconduction after an obser- vation period of 60 minutes or more was observed in 2  $\left(10\%\right)$  PVs. PV reconduction was successfully eliminated by additional RF applications  $(4.5\ \pm$  2.2 times) in all patients.

Appearance and Elimination of Dormant PV Conduction In 81 PVs isolated, 12 PVs exhibited spon- taneous reconduction within an observation pe- riod of  $66\,\pm\,55$  minutes. Among the remaining 69 PVs, 8  $\left(12\%\right)$ ) PVs had reconduction with the ATP injection (Fig. 2). Therefore,  $40\%$  of all recon- duction (8 of 20 PVs) was detected only with the use of ATP. All dormant PV reconductions were successfully eliminated by  $3.8\ \pm\ 1.2$  additional RF applications. After additional RF applications, complete PV disconnection was then conﬁrmed by repeat ATP injections under ISP infusion. AF was induced in three patients even after the comple- tion of PVI was conﬁrmed with the ATP injection.

These three patients (paroxysmal, two patients; persistent, one patient) were free from AF after the ablation procedure during a follow-up period of more than one year.

Patient Characteristics with and Without PV Reconduction A higher incidence of reconduction was ob- served in men (13/18) than in women (0/3) ( $\mathrm{~P~}<$  0.05). There was no difference in age, LA volume, LA volume index, or paroxysmal/permanent ratio of AF between groups with and without reconduc- tion (Table I).

 ${\mathsf{N S}}=$ s are mean ± SD. NS = not signiﬁcant.

## PV Characteristics with and Without PV Reconduction

RF energy delivered for the completion of PVI did not signiﬁcantly differ between PVs with sub- sequent rec $15811\,\pm\,8037$  those without ( $,16332\pm$  7243 J vs. 15811 ± 8037 J).  $22\pm5$ was no d $22\pm$   $5\,\mathrm{\mm})$ n PV ostium diamet $(63\,\pm\,13\,\mathrm{{\mm}}$  vs. $65~\pm$   $12\;\mathrm{mm})$  or circumference (63 ± 13 mm vs. 65 ± 12 mm) between PVs with and without reconduc- tion (Table II).

Discussion Major Findings This study demonstrated that a speciﬁc obser- vation period and the use of ATP were essential to detect early reconduction after PVI for AF dur- ing the procedure. This report shows that mean follow-up time was longer than 60 minutes and electrical activation of the PVs could be transiently reactivated only by ATP in  $40\%$  of PVs with recon- duction after successful PV antrum isolation with the interval of at least 30 minutes.

Cheema et al. showed the incidence and time course of PV reconduction after catheter ablation of AF during the procedure. They reported that PV reconduction was observed in 13  $\left(93\%\right)$  pa- tients and 26  $\left(50\%\right)$  veins after complete isola- tion of all PVs. The results revealed an extremely high rate of PV reconduction during the proce- dure. In the present study, the incidence of PV reconduction was less than in their report. The reason may be that we produced a continuous RF lesion using higher RF energy and a longer time. A higher RF energy and longer time applica- tion may increase the risk of fatal complications such as atrioesophageal ﬁstula. Therefore, we used esophageal monitoring to avoid esophageal damage.



Two previous reports described the preva- lence of acute PV reconduction during 60 minutes of observation. In the present study, reconduc- tion was detected over 60 minutes in two  $\left(10\%\right)$  veins. Thus, observation for over 60 minutes may be needed to detect reconduction after PVI during the procedure.

Mechanisms of ATP-Induced Reconduction In a previous report, RF application around the PVs was shown to produce local edema. Lo- cal edema may mask the gap of incomplete PVI due to nontransmural RF lesions, which may re- sult in the a false endpoint of PVI. Since some time is needed for improvement of local edema, the use of ATP may help detect incomplete RF lesions. The mechanism of ATP-induced transient PV re- conduction is not fully understood, but adenosine activates the outward potassium current via the purinergic  $\mathrm{A}_{1}$ -receptor, and the resulting hy- perpolarization of the cell membrane and shorten- ing of the action potential/refractory period may facilitate electronic conduction. Further investi- gation is necessary to clarify the mechanism of ATP induced reconduction.

Inducibility of Atrial Fibrillation Using Adenosine Triphosphate ATP sometimes induces AF, as previously re- ported. In the present study, AF was induced in three patients after PVI with the injection of ATP. Atienza et al. reported the effects of adenosine in- jection on the local activation frequency at differ- ent sites of the ﬁbrillating atria of patients before the ablation. In paroxysmal AF patients, adeno- sine increased activation frequency in the PV–LA junction. In persistent AF patients, the highest- frequency sources accelerated by adenosine were located in either atria but not at PV sites. The re- sults strongly suggest that AF is maintained by re-entrant sources, most likely located at the PV– LA junction in paroxysmal AF, whereas non-PV locations are more likely in persistent AF. In the present study, three patients who had AF induced after successful PVI were free from AF after the ablation procedure. Thus, inducibility of AF after PVI with ATP injection may be related to a non-PV trigger, but it may not be related to long-term AF recurrence.



Patient and PV Characteristics A higher incidence of reconduction was ob- served in men than in women  $(\mathrm{P\<\0.05})$ . Pre- vious studies have consistently shown that men have higher sympathetic activity, whereas women have higher parasympathetic activity. Sympa- thetic activity may be related to the incidence of reconduction.



There was no difference in age, LA volume, and volume index, or the paroxysmal/permanent ratio of AF between the recurrence and nonrecur- rence groups. Yamada et al. demonstrated that an increase in the RF energy needed for PVI was the only predictor of reconduction. However, RF en- ergy for initial PVI did not differ between PVs with reconduction and those without. The factors asso- ciated with reconduction may include myocardial wall thickness, the PV ostium diameter and cir- cumference, and difﬁculty in catheter manipula- tion because of anatomical limitations. It is likely that some of these factors were combined in many cases.

## Limitations

This study possesses several limitations. The ﬁrst is the reproducibility of the effects of ATP. Matsuo et al. assessed the reproducibility of the appearance of PV reconduction with ATP admin- istration. Three methods were compared for the

## References

1. Verma A, Kilicaslan F, Pisano E, Marrouche NF, Fanelli R, Brach- mann J, Geunther J, et al. Response of atrial ﬁbrillation to pul- monary vein antrum isolation is directly related to resumption and delay of pulmonary vein conduction. Circulation 2005; 112:627– 635.

2. Cappato R, Negroni S, Pecora D, Bentivegna S, Lupo PP, Carolei A, Esposito C, et al. Prospective assessment of late conduction recur- rence across radiofrequency lesions producing electrical disconnec- tion at the pulmonary vein ostium in patients with atrial ﬁbrillation. Circulation 2003; 108:1599–1604.

3. Gerstenfeld EP, Callans DJ, Dixit S, Zado E, Marchlinski FE. In- cidence and location of focal atrial ﬁbrillation triggers in pa- tients undergoing repeat pulmonary vein isolation: Implications for ablation strategies. J Cardiovasc Electrophysiol 2003; 14:685– 690.

4. Arentz T, Macle L, Kalusche D, Hocini M, Jais P, Shah D, Hais- saguerre M. “Dormant” pulmonary vein conduction revealed by adenosine after ostial radiofrequency catheter ablation. J Cardio- vasc Electrophysiol 2004; 15:1041–1047.

5. Matsuo S, Yamane T, Date T, Inada K, Kanzaki Y, Tokuda M, Shibayama K, et al. Reduction of AF recurrence after pulmonary vein isolation by eliminating ATP-induced transient venous re- conduction. J Cardiovasc Electrophysiol 2007; 18:704–708.

6. Kato R, Lickfett L, Meininger G, Dickfeld T, Wu R, Juang G, Angkeow P, et al. Pulmonary vein anatomy in patients under- going catheter ablation of atrial ﬁbrillation: Lessons learned by use of magnetic resonance imaging. Circulation 2003; 107:2004– 2010.

7. Yamada T, McElderry HT, Epstein AE, Plumb VJ, Kay GN. One- puncture, double-transseptal catheterization manoeuvre in the catheter ablation of atrial ﬁbrillation. Europace 2007; 9:487–489.

8. Yamada T, Okada T, Murakami Y. A novel catheter for simultaneous angiography of ipsilateral pulmonary veins. Europace 2007; 9:62– 63.

9. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama

induction of dormant PV conduction: ATP  $10\;\mathrm{mg}$ , ATP  $20~\mathrm{mg}$ , and ATP  $20~\mathrm{mg}$  during ISP infusion. Among their 10 cases, PV reconduction could be reproducibly induced only by ATP  $20~\mathrm{mg}$  admin- istration during ISP infusion. On the other hand, Hachiya et al. attempted to provoke PV reconduc- tion by injection of intravenous ATP 30 mg during intravenous isoproterenol infusion  $(1{-}3\ \mu\mathrm{g/min})$  after PVI. Thus, we used  $30~\mathrm{\mg}$  ATP during ISP infusion as the routine method to induce PV reconduction.

The second limitation involved the appropri- ate time to use ATP. We are not aware of any evidence that suggests that ATP should be used immediately after PVI or after a speciﬁc obser- vation period. Therefore, further study is needed to determine the appropriate time for the use of ATP.

Third, this study was neither large population nor randomized. Further examination is expected to be conﬁrmed the results in a larger patient col- lective with randomized assignment to the pro- posed technique versus a conventional one.

## Conclusion

The present ﬁndings suggest that a sufﬁcient observation period and the use of ATP are use- ful to detect reconduction after PVI during the procedure.

J, Yoshida Y, et al. Electrophysiologic pulmonary vein antrum iso- lation with a multielectrode basket catheter is feasible and effec- tive for curing paroxysmal atrial ﬁbrillation: Efﬁcacy of minimally extensive pulmonary vein isolation. Heart Rhythm 2006; 4:377– 384.

10. Cheema A, Dong J, Dalal D, Marine JE, Henrikson CA, Spragg D, Cheng A, et al. Incidence and time course of early recovery of pul- monary vein conduction after catheter ablation of atrial ﬁbrillation. J Cardiovasc Electrophysiol 2007; 18:387–391.

11. Pappone C, Oral H, Santinelli V, Vicedomini G, Lang CC, Manguso F, Torracca L, et al. Atrio-esophageal ﬁstula as a complication of percutaneous transcatheter ablation of atrial ﬁbrillation. Circulation 2004; 109:2724–2726.

12. Wang XH, Liu X, Sun YM, Gu JN, Shi HF, Zhou L, Hu W. Early identiﬁcation and treatment of PV re-connections: Role of observa- tion time and impact on clinical results of atrial ﬁbrillation ablation. Europace 2007; 9:481–486.

13. Okada T, Yamada T, Murakami Y, Yoshida N, Ninomiya Y, Shimizu T, Toyama J, et al. Prevalence and severity of left atrial edema de- tected by electron beam tomography early after pulmonary vein ablation. J Am Coll Cardiol 2007; 49:1436–1442.

14. Berlardinelli L, Linden J, Berne RM. The cardiac effects of adeno- sine. Prog Cardiovasc Dis 1989; 32:73–97.

15. Lerman B, Berlardinelli L. Cardiac electrophysiology of adenosine. Basic and clinical concepts. Circulation 1991; 83:1499–1509.

16. Belhassen B, Pelleg A, Shoshani D, Laniado S. Atrial ﬁbrillation induced by adenosine triphosphate. Am J Cardiol 1984; 53:1405– 1406.

17. Atienza F, Almendral J, Moreno J, Vaidyanathan R, Talkachou A, Kalifa J, Arenal A, et al. Activation of inward rectiﬁer potassium channels accelerates atrial ﬁbrillation in humans: Evidence for a reentrant mechanism. Circulation 2006; 114:2434–2442.

18. Jones PP, Snitker S, Skinner JS, Ravussin E. Gender differences in muscle sympathetic nerve activity: Effect of body fat distribution. Am J Physiol 1996; 270:E363–6.

19. Yamasaki Y, Kodama M, Matsuhisa M, Kishimoto M, Ozaki H, Tani A, Ueda N, et al. Diurnal heart rate variability in healthy sub- jects: Effects of aging and sex difference. Am J Physiol 1996; 271: H303–10.

20. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama J, Yoshida Y, et al. Incidence, location, and cause of recovery of electrical connections between the pulmonary veins and the left atrium after pulmonary vein isolation. Europace 2006; 8:182– 188.



21. Hachiya H, Hirao K, Takahashi A, Nagata Y, Suzuki K, Maeda S, Sasaki T, et al. Clinical implications of reconnection between the left atrium and isolated pulmonary veins provoked by adenosine triphosphate after extensive encircling pulmonary vein isolation. J Cardiovasc Electrophysiol 2007; 18:392–398.
# Usefulness of the Adenosine Triphosphate with a Sufﬁcient Observation Period for Detecting Reconduction after Pulmonary Vein Isolation

YUICHI NINOMIYA, M.D., YASUHISA IRIKI, M.D., SANEMASA ISHIDA, M.D., NAOYA OKETANI, M.D., TAKEHIKO MATSUSHITA, M.D., HITOSHI ICHIKI, M.D., HIDEKI OKUI, M.D., TETSURO KATAOKA, M.D., AKIKO YOSHIKAWA, M.D., MASAKAZU OGAWA, M.D., KEISHI SAIHARA, M.D., TAKURO SHINSATO, M.D., MASAAKI MIYATA, M.D., SHUICHI HAMASAKI, M.D., and CHUWA TEI, M.D. From the Department of Cardiovascular, Respiratory and Metabolic Medicine, Graduate School of Medical and Dental Sciences, Kagoshima University, Kagoshima, Japan Background: Although reconduction after pulmonary vein (PV) isolation is considered to play a key role in the recurrence of paroxysmal atrial ﬁbrillation  $(A F)$ , there have been few reports regarding the precise time course of early reconduction. Several studies have suggested that transient PV reconduction facilitated by adenosine may predict long-term  $A F$  recurrence. This study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods: $56\pm\,11$  ential PV antrum ablation was performed in  $a l l\;4\;P V s$ . After the completion of isolation, electrograms in each PV were repeatedly recorded ( $\left(1.98\pm0.57\right.$  times per PV) using a circular mapping catheter for an observation period of  $\ {87\pm29}$  was administered during isoproterenol infusion.

Results: PV electrical isolation was initially achieved in all 81 PVs. During the observation period, 12  $(15\%)$  $\left(15\%\right)P V s\;i n\ 10\left(48\%\right)$  of 21 patients exhibited spontaneous reconduction. Among the remaining 69 PVs, 8  $(12\%)$  additional PVs had reconduction with the use of ATP. All PV reconduction was successfully eliminated by  $4.5\pm2.2$  additional radiofrequency applications.

Conclusion: A sufﬁcient observation period and the use of ATP are useful to detect early reconduction after PV isolation. (PACE 2009; 32:1307–1312)

## adenosine triphosphate, atrial ﬁbrillation, observation period, pulmonary vein isolation, reconduction

Introduction Although reconduction of pulmonary veins (PVs) after electrical PV isolation (PVI) is consid- ered to play a key role in the recurrence of atrial ﬁbrillation (AF), there have been few reports regarding the precise time course of the reconduc- tion during the procedure. Arentz et al. recently re- ported that transient PV reconduction, which may predict long-term AF recurrence, could be facil- itated by adenosine. Matsuo et al. subsequently suggested that additional radiofrequency (RF) en- ergy application to eliminate transient PV recon- nection induced by adenosine triphosphate (ATP) reduced AF recurrence after PVI. The present study was designed to clarify the incidence and time course of early reconduction after PVI during the procedure and to conﬁrm whether the use of ATP after a certain observation period was useful to detect early reconduction after PVI.



Methods Study Population The study population consisted of 21 patients  years) with drug re- $56\pm11$  fractory symptomatic AF (paroxysmal, 14; persis- tent, 7 patients). Written informed consent was ob- tained from all patients. All antiarrhythmic drugs were discontinued for at least 5 half-lives before the study.

Preprocedure Measurements The left atrial (LA) dimension, volume, vol- ume index, and left ventricular ejection frac- tion (LVEF) were evaluated by transthoracic echocardiography.PVostiumdiametersand circumferences were measured on multidetector computed tomography (MDCT). The PV ostium was deﬁned as the point of inﬂection between the LA wall and the PV wall. The echocardiography and MDCT evaluators were blinded to the patient’s history. The LA dimension, LA volume, LA vol-  $68\pm18\,\mathrm{ml}$  $40\pm5\,\mathrm{mm}$   $39\pm9\;\mathrm{mL}/\mathrm{m}^{2}$  $67\pm9\%$  Electrophysiologic Study A7-Fhexapolarcatheterwasinserted transnasally into the esophagus. A 14-pole, 2- site mapping lumen catheter (Irvine Biomedical, Inc., Irvine, CA, USA) was positioned in the coro- nary sinus (CS) via the right jugular vein. The transseptal procedure was performed with in- tracardiac echocardiography guidance. Thereafter, intravenous heparin was administered to main- tain the activated clotting time between 250 and 300 seconds. Catheterization into the LA was per- formed using a 1-puncture and 2-sheath tech- nique. Two long vascular sheaths (SL0, St. Jude Medical, Minnetonka, MN, USA) were introduced into the LA. Simultaneous angiography of the ip- silateral 2 PVs using the 6-F Goodtec catheter (Goodman Co., Ltd, Nagoya, Japan) was performed to identify the PV ostia and antrum. PV map- ping was performed with a steerable 20-electrode catheter (OPTIMATM, Irvine Biomedical, Inc). If the catheter could not be deployed in the PVs, a 20-electrode, 15-mm-diameter, circular catheter (Lasso, Biosense Webster, Diamond Bar, CA, USA) was used for mapping those PVs. The catheter size was selected based on the predetermined mea- surement of the PV ostial diameter on angiogra- phy. Electrograms were recorded by the circular mapping catheter during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). When AF persisted during the electrophysiologic study, internal cardioversion using the CS catheter was performed to restore sinus rhythm, and the record- ing during appropriate rhythm was obtained for at least one beat.

  

 



PV Isolation The left superior PV (LSPV), left inferior PV (LIPV), right superior PV (RSPV), and right infe- rior PV (RIPV) were all targeted for isolation. Cir- cumferential PV antrum ablation targeting the PV antrum potentials was performed as previously described. RF energy (CABL-IT II, Japan Lifeline Co., Ltd, Tokyo, Japan) was delivered with a tar- get temperature of  $55^{\circ}\mathrm{C}$  and a maximum power output of  $30~\mathrm{~W~}$  for the LA posterior wall and 35 W elsewhere for 30 seconds using an 8-mm- tip catheter (Ablaze Fantasista, Japan Lifeline Co., Ltd, Tokyo, Japan). If a residual conduction gap was detected after the circumferential ablation, ad- ditional RF applications to the PV side next to the previous RF lesions were delivered. The endpoint of this technique was the completion of PV elec- trical disconnection.



Assessment of PV Electrical Isolation After isolation of each PV, electrograms in the PV were repeatedly recorded using the cir- cular mapping catheter. When reconduction was observed, additional ablation was performed un- til the PV was electrically isolated. After an ob- servation period of 30 minutes or more after the isolation of all 4 PVs, two consecutive series of ATP injections were performed to evaluate dor- mant PV conduction for the right and then the left ipsilateral 2 PVs. The presence/absence of PV conduction was evaluated using two circular map- ping catheters, which were positioned at the right and then the left ipsilateral PVs while the ma- neuver was performed. In all patients, ATP was rapidly injected under isoproterenol (ISP; 0.01–  $0.{\bar{0}}3\,\mathrm{mg/kg/min})$  infusion during sinus rhythm (for right PVs) or distal CS pacing (for left PVs). The ap- pearance of PV conduction was deﬁned as the tran- sient or continuous re-appearance of PV potentials associated with atrial depolarization (Fig. 1). If PV conduction re-appeared, additional RF energy was applied at the earliest PV activation site identiﬁed on the circular mapping catheter until complete PV disconnection was obtained. Complete PV dis- connection was then conﬁrmed by repeat ATP in- jections under ISP infusion.

## Patient Follow-Up

After the PVI procedure, all patients received periodic follow-up (at least once a month) in an outpatient clinic. The presence/absence of AF was evaluated by their symptoms, ECG recordings, and 24-hour ambulatory monitoring (1, 3, 6, and 12 months after the procedure). All patients who had symptoms without documentation of AF re- currence were given an event recorder (available for 5 days) to identify the cause of their symptoms. Freedom from AF was deﬁned as no AF with the patient off medication.

## Statistical Analysis

All values are expressed as the mean  $\pm$  standard deviation (SD). Statistical analysis was conducted using Mann-Whitney’s U test or  $\chi^{2}$  analysis. Differences with  $\mathrm{~P~}<\,\dot{0}.05$  were consid- ered to be statistically signiﬁcant.

Results A common left PV trunk that was observed in three patients was regarded as the LSPV in this study. Therefore, PVI was performed in 21 LSPVs, 18 LIPVs, 21 RSPVs, and 21 RIPVs in all 21 patients. After the completion of PVI, electro- grams in each PV were repeatedly recorded  $(1.98\pm$  0.57 times per PV) using the circular mapping  $\left(25\,\%\right)$ utes). $87\pm29$  PV reconduction was observed in 20 (25%) of 81 PVs in 13  $\left(62\%\right)$  patients after  $66\,\pm\,55$  minutes. Among these 13 patients, one patient had recon- duction in 3 PVs, ﬁve patients had reconduction in 2 PVs, and seven patients had reconduction in 1 PV. The numbers of reconductions observed in the LSPV, LIPV, RSPV, and RIPV were 6, 6, 5, and 3, respectively. Reconduction after an obser- vation period of 60 minutes or more was observed in 2  $\left(10\%\right)$  PVs. PV reconduction was successfully eliminated by additional RF applications  $(4.5\ \pm$  2.2 times) in all patients.

Appearance and Elimination of Dormant PV Conduction In 81 PVs isolated, 12 PVs exhibited spon- taneous reconduction within an observation pe- riod of  $66\,\pm\,55$  minutes. Among the remaining 69 PVs, 8  $\left(12\%\right)$ ) PVs had reconduction with the ATP injection (Fig. 2). Therefore,  $40\%$  of all recon- duction (8 of 20 PVs) was detected only with the use of ATP. All dormant PV reconductions were successfully eliminated by  $3.8\ \pm\ 1.2$  additional RF applications. After additional RF applications, complete PV disconnection was then conﬁrmed by repeat ATP injections under ISP infusion. AF was induced in three patients even after the comple- tion of PVI was conﬁrmed with the ATP injection.

These three patients (paroxysmal, two patients; persistent, one patient) were free from AF after the ablation procedure during a follow-up period of more than one year.

Patient Characteristics with and Without PV Reconduction A higher incidence of reconduction was ob- served in men (13/18) than in women (0/3) ( $\mathrm{~P~}<$  0.05). There was no difference in age, LA volume, LA volume index, or paroxysmal/permanent ratio of AF between groups with and without reconduc- tion (Table I).

 ${\mathsf{N S}}=$ s are mean ± SD. NS = not signiﬁcant.

## PV Characteristics with and Without PV Reconduction

RF energy delivered for the completion of PVI did not signiﬁcantly differ between PVs with sub- sequent rec $15811\,\pm\,8037$  those without ( $,16332\pm$  7243 J vs. 15811 ± 8037 J).  $22\pm5$ was no d $22\pm$   $5\,\mathrm{\mm})$ n PV ostium diamet $(63\,\pm\,13\,\mathrm{{\mm}}$  vs. $65~\pm$   $12\;\mathrm{mm})$  or circumference (63 ± 13 mm vs. 65 ± 12 mm) between PVs with and without reconduc- tion (Table II).

Discussion Major Findings This study demonstrated that a speciﬁc obser- vation period and the use of ATP were essential to detect early reconduction after PVI for AF dur- ing the procedure. This report shows that mean follow-up time was longer than 60 minutes and electrical activation of the PVs could be transiently reactivated only by ATP in  $40\%$  of PVs with recon- duction after successful PV antrum isolation with the interval of at least 30 minutes.

Cheema et al. showed the incidence and time course of PV reconduction after catheter ablation of AF during the procedure. They reported that PV reconduction was observed in 13  $\left(93\%\right)$  pa- tients and 26  $\left(50\%\right)$  veins after complete isola- tion of all PVs. The results revealed an extremely high rate of PV reconduction during the proce- dure. In the present study, the incidence of PV reconduction was less than in their report. The reason may be that we produced a continuous RF lesion using higher RF energy and a longer time. A higher RF energy and longer time applica- tion may increase the risk of fatal complications such as atrioesophageal ﬁstula. Therefore, we used esophageal monitoring to avoid esophageal damage.



Two previous reports described the preva- lence of acute PV reconduction during 60 minutes of observation. In the present study, reconduc- tion was detected over 60 minutes in two  $\left(10\%\right)$  veins. Thus, observation for over 60 minutes may be needed to detect reconduction after PVI during the procedure.

Mechanisms of ATP-Induced Reconduction In a previous report, RF application around the PVs was shown to produce local edema. Lo- cal edema may mask the gap of incomplete PVI due to nontransmural RF lesions, which may re- sult in the a false endpoint of PVI. Since some time is needed for improvement of local edema, the use of ATP may help detect incomplete RF lesions. The mechanism of ATP-induced transient PV re- conduction is not fully understood, but adenosine activates the outward potassium current via the purinergic  $\mathrm{A}_{1}$ -receptor, and the resulting hy- perpolarization of the cell membrane and shorten- ing of the action potential/refractory period may facilitate electronic conduction. Further investi- gation is necessary to clarify the mechanism of ATP induced reconduction.

Inducibility of Atrial Fibrillation Using Adenosine Triphosphate ATP sometimes induces AF, as previously re- ported. In the present study, AF was induced in three patients after PVI with the injection of ATP. Atienza et al. reported the effects of adenosine in- jection on the local activation frequency at differ- ent sites of the ﬁbrillating atria of patients before the ablation. In paroxysmal AF patients, adeno- sine increased activation frequency in the PV–LA junction. In persistent AF patients, the highest- frequency sources accelerated by adenosine were located in either atria but not at PV sites. The re- sults strongly suggest that AF is maintained by re-entrant sources, most likely located at the PV– LA junction in paroxysmal AF, whereas non-PV locations are more likely in persistent AF. In the present study, three patients who had AF induced after successful PVI were free from AF after the ablation procedure. Thus, inducibility of AF after PVI with ATP injection may be related to a non-PV trigger, but it may not be related to long-term AF recurrence.



Patient and PV Characteristics A higher incidence of reconduction was ob- served in men than in women  $(\mathrm{P\<\0.05})$ . Pre- vious studies have consistently shown that men have higher sympathetic activity, whereas women have higher parasympathetic activity. Sympa- thetic activity may be related to the incidence of reconduction.



There was no difference in age, LA volume, and volume index, or the paroxysmal/permanent ratio of AF between the recurrence and nonrecur- rence groups. Yamada et al. demonstrated that an increase in the RF energy needed for PVI was the only predictor of reconduction. However, RF en- ergy for initial PVI did not differ between PVs with reconduction and those without. The factors asso- ciated with reconduction may include myocardial wall thickness, the PV ostium diameter and cir- cumference, and difﬁculty in catheter manipula- tion because of anatomical limitations. It is likely that some of these factors were combined in many cases.

## Limitations

This study possesses several limitations. The ﬁrst is the reproducibility of the effects of ATP. Matsuo et al. assessed the reproducibility of the appearance of PV reconduction with ATP admin- istration. Three methods were compared for the

## References

1. Verma A, Kilicaslan F, Pisano E, Marrouche NF, Fanelli R, Brach- mann J, Geunther J, et al. Response of atrial ﬁbrillation to pul- monary vein antrum isolation is directly related to resumption and delay of pulmonary vein conduction. Circulation 2005; 112:627– 635.

2. Cappato R, Negroni S, Pecora D, Bentivegna S, Lupo PP, Carolei A, Esposito C, et al. Prospective assessment of late conduction recur- rence across radiofrequency lesions producing electrical disconnec- tion at the pulmonary vein ostium in patients with atrial ﬁbrillation. Circulation 2003; 108:1599–1604.

3. Gerstenfeld EP, Callans DJ, Dixit S, Zado E, Marchlinski FE. In- cidence and location of focal atrial ﬁbrillation triggers in pa- tients undergoing repeat pulmonary vein isolation: Implications for ablation strategies. J Cardiovasc Electrophysiol 2003; 14:685– 690.

4. Arentz T, Macle L, Kalusche D, Hocini M, Jais P, Shah D, Hais- saguerre M. “Dormant” pulmonary vein conduction revealed by adenosine after ostial radiofrequency catheter ablation. J Cardio- vasc Electrophysiol 2004; 15:1041–1047.

5. Matsuo S, Yamane T, Date T, Inada K, Kanzaki Y, Tokuda M, Shibayama K, et al. Reduction of AF recurrence after pulmonary vein isolation by eliminating ATP-induced transient venous re- conduction. J Cardiovasc Electrophysiol 2007; 18:704–708.

6. Kato R, Lickfett L, Meininger G, Dickfeld T, Wu R, Juang G, Angkeow P, et al. Pulmonary vein anatomy in patients under- going catheter ablation of atrial ﬁbrillation: Lessons learned by use of magnetic resonance imaging. Circulation 2003; 107:2004– 2010.

7. Yamada T, McElderry HT, Epstein AE, Plumb VJ, Kay GN. One- puncture, double-transseptal catheterization manoeuvre in the catheter ablation of atrial ﬁbrillation. Europace 2007; 9:487–489.

8. Yamada T, Okada T, Murakami Y. A novel catheter for simultaneous angiography of ipsilateral pulmonary veins. Europace 2007; 9:62– 63.

9. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama

induction of dormant PV conduction: ATP  $10\;\mathrm{mg}$ , ATP  $20~\mathrm{mg}$ , and ATP  $20~\mathrm{mg}$  during ISP infusion. Among their 10 cases, PV reconduction could be reproducibly induced only by ATP  $20~\mathrm{mg}$  admin- istration during ISP infusion. On the other hand, Hachiya et al. attempted to provoke PV reconduc- tion by injection of intravenous ATP 30 mg during intravenous isoproterenol infusion  $(1{-}3\ \mu\mathrm{g/min})$  after PVI. Thus, we used  $30~\mathrm{\mg}$  ATP during ISP infusion as the routine method to induce PV reconduction.

The second limitation involved the appropri- ate time to use ATP. We are not aware of any evidence that suggests that ATP should be used immediately after PVI or after a speciﬁc obser- vation period. Therefore, further study is needed to determine the appropriate time for the use of ATP.

Third, this study was neither large population nor randomized. Further examination is expected to be conﬁrmed the results in a larger patient col- lective with randomized assignment to the pro- posed technique versus a conventional one.

## Conclusion

The present ﬁndings suggest that a sufﬁcient observation period and the use of ATP are use- ful to detect reconduction after PVI during the procedure.

J, Yoshida Y, et al. Electrophysiologic pulmonary vein antrum iso- lation with a multielectrode basket catheter is feasible and effec- tive for curing paroxysmal atrial ﬁbrillation: Efﬁcacy of minimally extensive pulmonary vein isolation. Heart Rhythm 2006; 4:377– 384.

10. Cheema A, Dong J, Dalal D, Marine JE, Henrikson CA, Spragg D, Cheng A, et al. Incidence and time course of early recovery of pul- monary vein conduction after catheter ablation of atrial ﬁbrillation. J Cardiovasc Electrophysiol 2007; 18:387–391.

11. Pappone C, Oral H, Santinelli V, Vicedomini G, Lang CC, Manguso F, Torracca L, et al. Atrio-esophageal ﬁstula as a complication of percutaneous transcatheter ablation of atrial ﬁbrillation. Circulation 2004; 109:2724–2726.

12. Wang XH, Liu X, Sun YM, Gu JN, Shi HF, Zhou L, Hu W. Early identiﬁcation and treatment of PV re-connections: Role of observa- tion time and impact on clinical results of atrial ﬁbrillation ablation. Europace 2007; 9:481–486.

13. Okada T, Yamada T, Murakami Y, Yoshida N, Ninomiya Y, Shimizu T, Toyama J, et al. Prevalence and severity of left atrial edema de- tected by electron beam tomography early after pulmonary vein ablation. J Am Coll Cardiol 2007; 49:1436–1442.

14. Berlardinelli L, Linden J, Berne RM. The cardiac effects of adeno- sine. Prog Cardiovasc Dis 1989; 32:73–97.

15. Lerman B, Berlardinelli L. Cardiac electrophysiology of adenosine. Basic and clinical concepts. Circulation 1991; 83:1499–1509.

16. Belhassen B, Pelleg A, Shoshani D, Laniado S. Atrial ﬁbrillation induced by adenosine triphosphate. Am J Cardiol 1984; 53:1405– 1406.

17. Atienza F, Almendral J, Moreno J, Vaidyanathan R, Talkachou A, Kalifa J, Arenal A, et al. Activation of inward rectiﬁer potassium channels accelerates atrial ﬁbrillation in humans: Evidence for a reentrant mechanism. Circulation 2006; 114:2434–2442.

18. Jones PP, Snitker S, Skinner JS, Ravussin E. Gender differences in muscle sympathetic nerve activity: Effect of body fat distribution. Am J Physiol 1996; 270:E363–6.

19. Yamasaki Y, Kodama M, Matsuhisa M, Kishimoto M, Ozaki H, Tani A, Ueda N, et al. Diurnal heart rate variability in healthy sub- jects: Effects of aging and sex difference. Am J Physiol 1996; 271: H303–10.

20. Yamada T, Murakami Y, Okada T, Okamoto M, Shimizu T, Toyama J, Yoshida Y, et al. Incidence, location, and cause of recovery of electrical connections between the pulmonary veins and the left atrium after pulmonary vein isolation. Europace 2006; 8:182– 188.



21. Hachiya H, Hirao K, Takahashi A, Nagata Y, Suzuki K, Maeda S, Sasaki T, et al. Clinical implications of reconnection between the left atrium and isolated pulmonary veins provoked by adenosine triphosphate after extensive encircling pulmonary vein isolation. J Cardiovasc Electrophysiol 2007; 18:392–398.
tion of the great arteries: a morphologic study. Pediatr Cardiol 1983;4(1):15–24.

2. Antonellis  $\mathrm{J},$  Rabaouni A, Kostopulous  $\mathrm{K},$  Margaris  $\mathrm{N},$  Kranidis A, Salahas A. Single coronary artery from the right sinus of Valsalva, associated with absence of left anterior descending artery and an ostium secundum type atrial sep- tal defect: a rare combination. A case report. Angiology 1996;47: 621–5.

3. Yamanaka O, Hobbs RE. Coronary artery anomalies in 126,565  Cathet Cardiovasc Diag 1990;12:28–40.

4. Click RL, Holmes DR, Vletstra RE, Kosinski AS, Kronmal RA. Anomalous coronary arteries: location, degree of atheroscle- rosis and effect on survival: a report from the Coronary Artery Surgery Study. J Am Coll Cardiol 1989;13:513–7.

5. Shirani J, Roberts WC. Solitary coronary ostium in the aorta in the absence of other major congenital cardiovascular anoma- lies. J Am Coll Cardiol 1993;21:137–43.

6. Turkay C, Golbasi I, Bayezid O. A single coronary artery from  Acta Cardiol57(5):377–9.

7. Benge W, Martins JB, Funk DC. Morbidity associated with anomalousoriginoftherightcoronaryarteryfromtheleftsinus of Valsalva. Am Heart J 1980;99:96–100.

8. Taylor AJ, Rogan KM, Virmani R. Sudden cardiac death associ- ated with isolated congenital coronary artery anomalies. J Am Coll Cardiol20:640–7.

9. Maron BJ. Sudden death in young athletes. N Engl $J$  Med 2003;349:1064–75.

# Coronary Artery Perforation Successfully Treated With Tris-acryl Gelatin Microsphere Embolisation

Andrew C.Y. To, MBChB a,∗, Seifeddin S. El-Jack, $\mathbf{MD}^{\mathtt{b}}.$ , Mark W.I. Webster, MD a and James T. Stewart, MD a

 The Green Lane Cardiovascular Service, Auckland City Hospital, Auckland, New Zealand  Department of Cardiology, North Shore Hospital, Auckland, New Zealand We describe a case of coronary artery perforation in a 76-year-old man, successfully treated by tris-acryl gelatin microsphere embolisation. This novel interventional embolic material is used in interventional radiology for arterial embolisation. We believe that this is the ﬁrst report of its use for a coronary artery perforation. (Heart, Lung and Circulation 2008;17:417–436)  $@$  2007 Australasian Society of Cardiac and Thoracic Surgeons and the Cardiac Society of Australia and New Zealand. Published by Elsevier Inc. All rights reserved.



### Keywords. Coronary perforation; Therapeutic embolisation; Cardiac tamponade; Coronary occlusion; Tris-acryl gelatin microspheres

## Introduction

Coronary artery perforation during percutaneous inter- ventions is relatively rare but associated with a high incidence of cardiac tamponade, myocardial infarction and death. While improvements in procedural tech- nique and equipment have reduced the incidence in routine cases, many complex cases with a higher risk of complications are now being treated percutaneously.

Percutaneous management strategies for coronary perfo- ration include prolonged balloon inﬂation, deployment of covered stents and arterial embolisation. We report a case of perforation managed successfully with tris-acryl gelatin microspheres embolisation.

Received 24 March 2007; received in revised form 17 May 2007; accepted 13 June 2007; available online 30 July 2007

## Case Presentation

A 76-year-old man presented with a non-ST elevation myocardial infarction (peak troponin-T  $8.37\,\upmu\mathrm{g/L}$  [nor- mal  ${<}0.031^{\circ}$ ). He was treated with aspirin, clopidogrel, a beta-blocker, an ACE-inhibitor and a statin. He was enrolled in the EARLY-ACS [Early glycoprotein IIb/IIIa inhibition in non-ST-segment elevation Acute Coronary Syndrome] trial, receiving either intravenous eptiﬁbatide or placebo. Coronary angiography showed a chronic occlusion of the proximal left circumﬂex artery  $(C x)$  with TIMI-I ﬂow, a severe mid-left anterior descending artery (LAD) stenosis and mild right coronary artery disease (Fig. 1a).

After administration of 8000 units of heparin, the mid-LAD stenosis was uneventfully treated using a paclitaxel-eluting stent. The  $C x$  lesion was crossed with a High Torgue Intermediate guidewire (Guidant, Santa Clara, CA) that appeared to move freely in the expected course of the distal LCx. After pre-dilatation with a  $1.25\,\mathrm{mm}$  balloon, limited extravasation of contrast from the mid- $C x$  into the pericardium (type 2 perforation) was detected (Fig. 1b). Percutaneous intervention was aborted but the trial medication (eptiﬁbatide or placebo) was con- tinued. The patient returned to the ward pain-free and haemodynamically stable.



Two hours later, the patient’s condition deteriorated rapidly with reduced level of consciousness and no recordable blood pressure. His trial medication was discontinued. Transthoracic echocardiography demon- strated a large pericardial effusion with tamponade. Emergency pericardiocentesis drained  $320\,\mathrm{ml}$  of blood, leading to immediate clinical improvement with return of systolic blood pressure to  $130\,\mathrm{mmHg}$ . He did not require intubation.

Repeat angiography demonstrated continuous ﬂow of contrast into the pericardial cavity (type 3 perforation) at the level of the mid- $C x$  (Fig. 1c). A  $2.0\,\mathrm{mm}$  over-the- wire balloon was positioned in the proximal  $C x$  and was inﬂated to 8 atmospheres, occluding the vessel. Because balloon occlusion alone did not seal the perforation, 3 ml of EmboGold $300–500\,\upmu\mathrm{m}$  microspheres (Biosphere Medical, Roissy, Paris), suspended 1:1 in diluted contrast solution, were slowly injected distally through the central balloon lumen. Balloon inﬂation was maintained for $8\,\mathrm{min}$  to pre- vent reﬂux of microspheres into the LAD and aorta. As the artery was occluded distal to the perforation, there was no risk of microspheres entering distal collaterals. Repeat angiography revealed complete occlusion of the $C x$  at the embolisation site with no further leakage of contrast into the pericardial space (Fig. 1d).



Post-embolisation, a further $270\,\mathrm{ml}$  of blood was drained over $24\,\mathrm{h}$ . Subsequent echocardiography showed a small circumferential rim of ﬂuid, with no sign of tamponade.

## Discussion

Management of coronary artery perforation includes reversal of anticoagulation, pericardiocentesis for car- diac tamponade and, when necessary, deﬁnitive treatment of the perforation. While emergency surgery may be required, percutaneous techniques are often successful.

Polytetraﬂuoroethylene (PTFE)-covered stents and autol- ogous vein grafts covered stents are useful in larger vessels in which the delivery system can be tracked to the perforation site. For distal vessel guidewire perfora- tions and perforations of occluded or small calibre vessels, embolisation with microcoils, gelfoam, polyvinyl alco- hol (PVA) particles have been reported to be effective. The published literature comprises case reports, with no method clearly demonstrated to be superior. In this case, it was felt that the vessel was too small for coil embolisation or covered-stent placement. Our previous experience with gelfoam was that injection through the central lumen of a standard angioplasty balloon catheter could sometimes be difﬁcult. Bench testing with microspheres, however, had shown that that they were easily injected down the   $0.014^{\prime\prime}$ catheter and appeared suitable for sealing this perforation.

Coronary perforation in our patient was related to the attempted crossing of a chronically occluded ves- sel in the setting of intense anti-platelet therapy. In retrospect, attempted percutaneous intervention to a non- infarct related chronic occlusion might better have been undertaken as a second-stage procedure. Percutaneous intervention was promptly aborted upon the detection of contrast extravasation outside the vessel. Continuation of the trial medication (eptiﬁbatide or placebo) may have contributed to continued bleeding into the pericardium. Microspheres, a novel embolisation agent, were chosen because of potential limitations of other techniques in treating this particular perforation.

Microspheres, containing an acrylic co-polymer (tris- acryl) cross-linked with gelatin, were developed as an alternative to PVA particles in interventional radiol- ogy. They are designed to be hydrophilic, deformable, uniformly spherical particles with a narrow range of par- ticle diameters, contrasting with PVA particles which are irregularly shaped with sharp and jagged edges. A  $300{-}500\,\upmu\mathrm{m}$  suspension contains approximately 10,000 microspheres per millilitre. Problems with PVA parti- cles, such as particle aggregation, hub accumulation and catheter occlusion, are not observed with microspheres. Particulate penetration into a vascular lesion is improved because of avoidance of aggregate formation. Histo- logical studies have demonstrated a mild-to-moderate inﬂammatory reaction around the microspheres with no evidence of degradation observed in specimens more than 6 months after embolisation. Previous studies have demonstrated a tight correlation between the diam- eter of the microspheres injected and the size of the occluded vessel. Therefore, it is likely that vessels of  or larger were occluded by the microspheres, $300\,\upmu\mathrm{m}$  with thrombus formation also contributing to the total occlusion of the LCx.  $300{-}500{-}{\upmu\mathrm{m}}$  microspheres were chosen in our case, based upon the diameter of the per- forated artery and the minimum inner diameter of the   $0.014^{\prime\prime}$ catheter. The elastic properties of the microspheres allow for temporary compression of up to  $33\%,$  facil- itating the passage of microspheres through injection catheters.



To our knowledge, this is the ﬁrst case report of coronary artery perforation treated successfully by transcatheter injection of tris-acryl gelatin microspheres. While there is a range of options for managing coronary artery per- foration, the use of microsphere embolisation appeared effective in this case and should be considered part of the armoury for managing this serious complication of percutaneous coronary intervention.

## References

1. Fasseas P, Orford JL, Panetta CJ, Bell MR, Denktas AE, Lennon RJ, Holmes DR, Berger PB. Incidence, correlates, manage- ment, and clinical outcome of coronary perforation: analysis  Am Heart J 2004;147(1):140–5. 2. Ellis SG, Ajluni S, Arnold  $\mathrm{A}Z,$  Popma  $\mathrm{J},$  Bittl JA, Eigler  $\mathrm{NL},$  Cowley MJ, Raymond RE, Saﬁan RD, Whitlow PL. Increased coronary perforation in the new device era. Inci- dence, classiﬁcation, management, and outcome. Circulation 1994;90(6):2725–30.

3. Gruberg L, Pinnow E, Flood R, Bonnet $\mathrm{Y,}$  Tebeica M, Waks- man R, Satler LF, Pichard AD, Kent KM, Leon MB, Lindsay Jr J. Incidence, management, and outcome of coronary artery perforation during percutaneous coronary intervention. Am J Cardiol86(6):680–2. A8.

4. Gunning MG, Williams  $\mathrm{IL},$  Jewitt DE, Shah AM, Wain- wright RJ, Thomas MR. Coronary artery perforation during percutaneous intervention: incidence and outcome. Heart 2002;88(5):495–8.

5. Giugliano RP, Newby LK, Harrington RA, Gibson CM, Van de Werf F, Armstrong P, Montalescot G, Gilbert J, Strony JT, Califf RM, Braunwald E. The early glycoprotein IIb/IIIa inhibition in non-ST-segment elevation acute coronary syndrome (EARLY ACS) trial: a randomized placebo-controlled trial evaluating the clinical beneﬁts of early front-loaded eptiﬁbatide in the treatment of patients with non-ST-segment elevation acute coronary syndrome – study design and rationale. Am Heart J 2005;149(6):994–1002.

6. Lansky AJ, Yang YM, Khan Y, Costa RA, Pietras  $C,$  Tsuchiya Y, Cristea E, Collins M, Mehran R, Dangas GD, Moses JW, Leon MB, Stone GW. Treatment of coronary artery perfora- tions complicating percutaneous coronary intervention with a polytetraﬂuoroethylene-covered stent graft. Am J Cardiol 2006;98(3):370–4.

7. Colombo A, Itoh A, Di Mario  $C,$  Maiello $\mathrm{L},$  Arena V, Blengino S, Briati P, Ferraro M, Di Francesco $\mathrm{L},$  Martini G. Successful closure of a coronary vessel rupture with a vein graft stent: case report. Catheter Cardiovasc Diagn 1996;38(2):172–4.

8. Assali AR, Moustapha A, Sdringola S, Rihner M, Smalling RW. Successful treatment of coronary artery perforation in an abciximab-treated patient by microcoil embolization. Catheter Cardiovasc Interv51(4):487–9.

9. Dixon SR, Webster MW, Ormiston JA, Wattie WJ, Ham- mettCJ.Gelfoamembolizationofadistalcoronary arteryguidewireperforation.CatheterCardiovascInterv 2000;49(2):214–7.

10. Yoo BS, Yoon J, Lee SH, Kim JY, Lee HH, Ko JY, Lee BK, Hwang SO, Choe KH. Guidewire-induced coronary artery perfora- tion treated with transcatheter injection of polyvinyl alcohol  Catheter Cardiovasc Interv52(2):231–4. 11. Derdeyn CP, Graves VB, Salamat MS, Rappe A. Collagen- coated acrylic microspheres for embolotherapy: in vivo and in  AJNR Am J Neuroradiol18(4):647–53.



12. Laurent A, Wassef M, Chapot R, Houdart E, Merland JJ. Location of vessel occlusion of calibrated tris-acryl gelatin microspheres for tumor and arteriovenous malformation embolization. $J$  Vasc Interv Radiol 2004;15(5):491–6.

# Very Late Stent Fracture Associated With a Sirolimus-eluting Stent

D.P. Leong, MBBS (Hons.) , B.K. Dundon, FRACP, R. Puri, MBBS and R.A.S. Yeend, FRACP

Cardiovascular Investigation Unit, Royal Adelaide Hospital, North Terrace, Adelaide, SA 5000, Australia Late stent thrombosis (>1 year after implantation) is a recognised complication involving drug-eluting stents. Stent fracture is increasingly being reported as a complication of drug-eluting stent, and in particular sirolimus-eluting stent use. We report the case of very late sirolimus-eluting stent fracture resulting in an acute coronary syndrome. This case report highlights the need for greater awareness and post-marketing surveillance to detect a potential mechanism for late stent thrombosis in the drug-eluting stent era. (Heart, Lung and Circulation 2008;17:417–436)  $\copyright$  2007 Australasian Society of Cardiac and Thoracic Surgeons and the Cardiac Society of Australia and New Zealand. Published by Elsevier Inc. All rights reserved.



### Keywords. Stent fracture; Stent thrombosis; Sirolimus-eluting stent; Drug-eluting stent

Introduction T he use of drug-eluting stents has been demonstrated to lower in-stent restenosis (ISR) rates, however the incidence of late complications such as stent thrombosis has raised concern over their use. In addition, there has been increasing awareness of stent fracture as a potential complication following drug-eluting stent implantation. We report a case of sirolimus-eluting stent (SES) fracture precipitating acute coronary syndrome more than 3 years after stent deployment.

## Case History

A 65-year-old man underwent coronary angiography for investigation of exertional angina. This conﬁrmed crit- ical lesions in the ﬁrst obtuse marginal branch of the circumﬂex artery (Fig. 1) with minor disease elsewhere and normal left ventriculography. Percutaneous coronary intervention to the culprit lesion was undertaken. The lesion was pre-dilated with $2.0\,\mathrm{mm}\times10\,\mathrm{mm}$  conventional angioplasty balloon to  $12\,\mathrm{atm.},$  followed by deployment of a $2.75\,\mathrm{mm}\times30\,\mathrm{mm}\;\mathrm{SE}$  result (Fig. 2).

Received 31 March 2007; received in revised form 29 April 2007; accepted 6 June 2007; available online 28 August 2007 The patient had an uneventful recovery and remained symptom free for almost 3.5 years, before re-presenting with a troponin positive acute coronary syndrome and electrocardiographic evidence of myocardial ischaemia in the lateral leads. Coronary angiography was repeated, revealing complete fracture of the previous SES near the midpoint. There was clear displacement of the stent segments from each other, resulting in a kink within the exposed artery (Fig. 3). The other coronary arteries remained unchanged. The stent fracture was treated with direct deployment of a  $3.0\,\mathrm{mm}\times13\,\mathrm{mm}$  SES to  $3.5\,\mathrm{mm}\times8\,\mathrm{mm}$  Powersail balloon to $12\,\mathrm{atm.},$  with an excellent ﬁnal result (Fig. 4). The patient remained well at 12-month follow-up, with a negative exercise treadmill test featuring normal exercise capacity.



## Discussion

Despite substantial clinical experience with drug-eluting stents in randomised trials and post-marketing reg- istry surveillance, coronary stent fracture has not been highlighted as a signiﬁcant clinical issue in the pub- lished literature. Nonetheless, awareness of this possible complication has recently begun to emerge with the publication of isolated case reports and small series. One such series suggested the prevalence of stent frac- ture may be as high as  $1.9\bar{\%},1$  with stent fracture occurringin10of530patientsundergoingclini- cally indicated repeat coronary angiography. All of these cases of stent fracture involved sirolimus-eluting
tion of the great arteries: a morphologic study. Pediatr Cardiol 1983;4(1):15–24.

2. Antonellis  $\mathrm{J},$  Rabaouni A, Kostopulous  $\mathrm{K},$  Margaris  $\mathrm{N},$  Kranidis A, Salahas A. Single coronary artery from the right sinus of Valsalva, associated with absence of left anterior descending artery and an ostium secundum type atrial sep- tal defect: a rare combination. A case report. Angiology 1996;47: 621–5.

3. Yamanaka O, Hobbs RE. Coronary artery anomalies in 126,565  Cathet Cardiovasc Diag 1990;12:28–40.

4. Click RL, Holmes DR, Vletstra RE, Kosinski AS, Kronmal RA. Anomalous coronary arteries: location, degree of atheroscle- rosis and effect on survival: a report from the Coronary Artery Surgery Study. J Am Coll Cardiol 1989;13:513–7.

5. Shirani J, Roberts WC. Solitary coronary ostium in the aorta in the absence of other major congenital cardiovascular anoma- lies. J Am Coll Cardiol 1993;21:137–43.

6. Turkay C, Golbasi I, Bayezid O. A single coronary artery from  Acta Cardiol57(5):377–9.

7. Benge W, Martins JB, Funk DC. Morbidity associated with anomalousoriginoftherightcoronaryarteryfromtheleftsinus of Valsalva. Am Heart J 1980;99:96–100.

8. Taylor AJ, Rogan KM, Virmani R. Sudden cardiac death associ- ated with isolated congenital coronary artery anomalies. J Am Coll Cardiol20:640–7.

9. Maron BJ. Sudden death in young athletes. N Engl $J$  Med 2003;349:1064–75.

# Coronary Artery Perforation Successfully Treated With Tris-acryl Gelatin Microsphere Embolisation

Andrew C.Y. To, MBChB a,∗, Seifeddin S. El-Jack, $\mathbf{MD}^{\mathtt{b}}.$ , Mark W.I. Webster, MD a and James T. Stewart, MD a

 The Green Lane Cardiovascular Service, Auckland City Hospital, Auckland, New Zealand  Department of Cardiology, North Shore Hospital, Auckland, New Zealand We describe a case of coronary artery perforation in a 76-year-old man, successfully treated by tris-acryl gelatin microsphere embolisation. This novel interventional embolic material is used in interventional radiology for arterial embolisation. We believe that this is the ﬁrst report of its use for a coronary artery perforation. (Heart, Lung and Circulation 2008;17:417–436)  $@$  2007 Australasian Society of Cardiac and Thoracic Surgeons and the Cardiac Society of Australia and New Zealand. Published by Elsevier Inc. All rights reserved.



### Keywords. Coronary perforation; Therapeutic embolisation; Cardiac tamponade; Coronary occlusion; Tris-acryl gelatin microspheres

## Introduction

Coronary artery perforation during percutaneous inter- ventions is relatively rare but associated with a high incidence of cardiac tamponade, myocardial infarction and death. While improvements in procedural tech- nique and equipment have reduced the incidence in routine cases, many complex cases with a higher risk of complications are now being treated percutaneously.

Percutaneous management strategies for coronary perfo- ration include prolonged balloon inﬂation, deployment of covered stents and arterial embolisation. We report a case of perforation managed successfully with tris-acryl gelatin microspheres embolisation.

Received 24 March 2007; received in revised form 17 May 2007; accepted 13 June 2007; available online 30 July 2007

## Case Presentation

A 76-year-old man presented with a non-ST elevation myocardial infarction (peak troponin-T  $8.37\,\upmu\mathrm{g/L}$  [nor- mal  ${<}0.031^{\circ}$ ). He was treated with aspirin, clopidogrel, a beta-blocker, an ACE-inhibitor and a statin. He was enrolled in the EARLY-ACS [Early glycoprotein IIb/IIIa inhibition in non-ST-segment elevation Acute Coronary Syndrome] trial, receiving either intravenous eptiﬁbatide or placebo. Coronary angiography showed a chronic occlusion of the proximal left circumﬂex artery  $(C x)$  with TIMI-I ﬂow, a severe mid-left anterior descending artery (LAD) stenosis and mild right coronary artery disease (Fig. 1a).

After administration of 8000 units of heparin, the mid-LAD stenosis was uneventfully treated using a paclitaxel-eluting stent. The  $C x$  lesion was crossed with a High Torgue Intermediate guidewire (Guidant, Santa Clara, CA) that appeared to move freely in the expected course of the distal LCx. After pre-dilatation with a  $1.25\,\mathrm{mm}$  balloon, limited extravasation of contrast from the mid- $C x$  into the pericardium (type 2 perforation) was detected (Fig. 1b). Percutaneous intervention was aborted but the trial medication (eptiﬁbatide or placebo) was con- tinued. The patient returned to the ward pain-free and haemodynamically stable.



Two hours later, the patient’s condition deteriorated rapidly with reduced level of consciousness and no recordable blood pressure. His trial medication was discontinued. Transthoracic echocardiography demon- strated a large pericardial effusion with tamponade. Emergency pericardiocentesis drained  $320\,\mathrm{ml}$  of blood, leading to immediate clinical improvement with return of systolic blood pressure to  $130\,\mathrm{mmHg}$ . He did not require intubation.

Repeat angiography demonstrated continuous ﬂow of contrast into the pericardial cavity (type 3 perforation) at the level of the mid- $C x$  (Fig. 1c). A  $2.0\,\mathrm{mm}$  over-the- wire balloon was positioned in the proximal  $C x$  and was inﬂated to 8 atmospheres, occluding the vessel. Because balloon occlusion alone did not seal the perforation, 3 ml of EmboGold $300–500\,\upmu\mathrm{m}$  microspheres (Biosphere Medical, Roissy, Paris), suspended 1:1 in diluted contrast solution, were slowly injected distally through the central balloon lumen. Balloon inﬂation was maintained for $8\,\mathrm{min}$  to pre- vent reﬂux of microspheres into the LAD and aorta. As the artery was occluded distal to the perforation, there was no risk of microspheres entering distal collaterals. Repeat angiography revealed complete occlusion of the $C x$  at the embolisation site with no further leakage of contrast into the pericardial space (Fig. 1d).



Post-embolisation, a further $270\,\mathrm{ml}$  of blood was drained over $24\,\mathrm{h}$ . Subsequent echocardiography showed a small circumferential rim of ﬂuid, with no sign of tamponade.

## Discussion

Management of coronary artery perforation includes reversal of anticoagulation, pericardiocentesis for car- diac tamponade and, when necessary, deﬁnitive treatment of the perforation. While emergency surgery may be required, percutaneous techniques are often successful.

Polytetraﬂuoroethylene (PTFE)-covered stents and autol- ogous vein grafts covered stents are useful in larger vessels in which the delivery system can be tracked to the perforation site. For distal vessel guidewire perfora- tions and perforations of occluded or small calibre vessels, embolisation with microcoils, gelfoam, polyvinyl alco- hol (PVA) particles have been reported to be effective. The published literature comprises case reports, with no method clearly demonstrated to be superior. In this case, it was felt that the vessel was too small for coil embolisation or covered-stent placement. Our previous experience with gelfoam was that injection through the central lumen of a standard angioplasty balloon catheter could sometimes be difﬁcult. Bench testing with microspheres, however, had shown that that they were easily injected down the   $0.014^{\prime\prime}$ catheter and appeared suitable for sealing this perforation.

Coronary perforation in our patient was related to the attempted crossing of a chronically occluded ves- sel in the setting of intense anti-platelet therapy. In retrospect, attempted percutaneous intervention to a non- infarct related chronic occlusion might better have been undertaken as a second-stage procedure. Percutaneous intervention was promptly aborted upon the detection of contrast extravasation outside the vessel. Continuation of the trial medication (eptiﬁbatide or placebo) may have contributed to continued bleeding into the pericardium. Microspheres, a novel embolisation agent, were chosen because of potential limitations of other techniques in treating this particular perforation.

Microspheres, containing an acrylic co-polymer (tris- acryl) cross-linked with gelatin, were developed as an alternative to PVA particles in interventional radiol- ogy. They are designed to be hydrophilic, deformable, uniformly spherical particles with a narrow range of par- ticle diameters, contrasting with PVA particles which are irregularly shaped with sharp and jagged edges. A  $300{-}500\,\upmu\mathrm{m}$  suspension contains approximately 10,000 microspheres per millilitre. Problems with PVA parti- cles, such as particle aggregation, hub accumulation and catheter occlusion, are not observed with microspheres. Particulate penetration into a vascular lesion is improved because of avoidance of aggregate formation. Histo- logical studies have demonstrated a mild-to-moderate inﬂammatory reaction around the microspheres with no evidence of degradation observed in specimens more than 6 months after embolisation. Previous studies have demonstrated a tight correlation between the diam- eter of the microspheres injected and the size of the occluded vessel. Therefore, it is likely that vessels of  or larger were occluded by the microspheres, $300\,\upmu\mathrm{m}$  with thrombus formation also contributing to the total occlusion of the LCx.  $300{-}500{-}{\upmu\mathrm{m}}$  microspheres were chosen in our case, based upon the diameter of the per- forated artery and the minimum inner diameter of the   $0.014^{\prime\prime}$ catheter. The elastic properties of the microspheres allow for temporary compression of up to  $33\%,$  facil- itating the passage of microspheres through injection catheters.



To our knowledge, this is the ﬁrst case report of coronary artery perforation treated successfully by transcatheter injection of tris-acryl gelatin microspheres. While there is a range of options for managing coronary artery per- foration, the use of microsphere embolisation appeared effective in this case and should be considered part of the armoury for managing this serious complication of percutaneous coronary intervention.

## References

1. Fasseas P, Orford JL, Panetta CJ, Bell MR, Denktas AE, Lennon RJ, Holmes DR, Berger PB. Incidence, correlates, manage- ment, and clinical outcome of coronary perforation: analysis  Am Heart J 2004;147(1):140–5. 2. Ellis SG, Ajluni S, Arnold  $\mathrm{A}Z,$  Popma  $\mathrm{J},$  Bittl JA, Eigler  $\mathrm{NL},$  Cowley MJ, Raymond RE, Saﬁan RD, Whitlow PL. Increased coronary perforation in the new device era. Inci- dence, classiﬁcation, management, and outcome. Circulation 1994;90(6):2725–30.

3. Gruberg L, Pinnow E, Flood R, Bonnet $\mathrm{Y,}$  Tebeica M, Waks- man R, Satler LF, Pichard AD, Kent KM, Leon MB, Lindsay Jr J. Incidence, management, and outcome of coronary artery perforation during percutaneous coronary intervention. Am J Cardiol86(6):680–2. A8.

4. Gunning MG, Williams  $\mathrm{IL},$  Jewitt DE, Shah AM, Wain- wright RJ, Thomas MR. Coronary artery perforation during percutaneous intervention: incidence and outcome. Heart 2002;88(5):495–8.

5. Giugliano RP, Newby LK, Harrington RA, Gibson CM, Van de Werf F, Armstrong P, Montalescot G, Gilbert J, Strony JT, Califf RM, Braunwald E. The early glycoprotein IIb/IIIa inhibition in non-ST-segment elevation acute coronary syndrome (EARLY ACS) trial: a randomized placebo-controlled trial evaluating the clinical beneﬁts of early front-loaded eptiﬁbatide in the treatment of patients with non-ST-segment elevation acute coronary syndrome – study design and rationale. Am Heart J 2005;149(6):994–1002.

6. Lansky AJ, Yang YM, Khan Y, Costa RA, Pietras  $C,$  Tsuchiya Y, Cristea E, Collins M, Mehran R, Dangas GD, Moses JW, Leon MB, Stone GW. Treatment of coronary artery perfora- tions complicating percutaneous coronary intervention with a polytetraﬂuoroethylene-covered stent graft. Am J Cardiol 2006;98(3):370–4.

7. Colombo A, Itoh A, Di Mario  $C,$  Maiello $\mathrm{L},$  Arena V, Blengino S, Briati P, Ferraro M, Di Francesco $\mathrm{L},$  Martini G. Successful closure of a coronary vessel rupture with a vein graft stent: case report. Catheter Cardiovasc Diagn 1996;38(2):172–4.

8. Assali AR, Moustapha A, Sdringola S, Rihner M, Smalling RW. Successful treatment of coronary artery perforation in an abciximab-treated patient by microcoil embolization. Catheter Cardiovasc Interv51(4):487–9.

9. Dixon SR, Webster MW, Ormiston JA, Wattie WJ, Ham- mettCJ.Gelfoamembolizationofadistalcoronary arteryguidewireperforation.CatheterCardiovascInterv 2000;49(2):214–7.

10. Yoo BS, Yoon J, Lee SH, Kim JY, Lee HH, Ko JY, Lee BK, Hwang SO, Choe KH. Guidewire-induced coronary artery perfora- tion treated with transcatheter injection of polyvinyl alcohol  Catheter Cardiovasc Interv52(2):231–4. 11. Derdeyn CP, Graves VB, Salamat MS, Rappe A. Collagen- coated acrylic microspheres for embolotherapy: in vivo and in  AJNR Am J Neuroradiol18(4):647–53.



12. Laurent A, Wassef M, Chapot R, Houdart E, Merland JJ. Location of vessel occlusion of calibrated tris-acryl gelatin microspheres for tumor and arteriovenous malformation embolization. $J$  Vasc Interv Radiol 2004;15(5):491–6.

# Very Late Stent Fracture Associated With a Sirolimus-eluting Stent

D.P. Leong, MBBS (Hons.) , B.K. Dundon, FRACP, R. Puri, MBBS and R.A.S. Yeend, FRACP

Cardiovascular Investigation Unit, Royal Adelaide Hospital, North Terrace, Adelaide, SA 5000, Australia Late stent thrombosis (>1 year after implantation) is a recognised complication involving drug-eluting stents. Stent fracture is increasingly being reported as a complication of drug-eluting stent, and in particular sirolimus-eluting stent use. We report the case of very late sirolimus-eluting stent fracture resulting in an acute coronary syndrome. This case report highlights the need for greater awareness and post-marketing surveillance to detect a potential mechanism for late stent thrombosis in the drug-eluting stent era. (Heart, Lung and Circulation 2008;17:417–436)  $\copyright$  2007 Australasian Society of Cardiac and Thoracic Surgeons and the Cardiac Society of Australia and New Zealand. Published by Elsevier Inc. All rights reserved.



### Keywords. Stent fracture; Stent thrombosis; Sirolimus-eluting stent; Drug-eluting stent

Introduction T he use of drug-eluting stents has been demonstrated to lower in-stent restenosis (ISR) rates, however the incidence of late complications such as stent thrombosis has raised concern over their use. In addition, there has been increasing awareness of stent fracture as a potential complication following drug-eluting stent implantation. We report a case of sirolimus-eluting stent (SES) fracture precipitating acute coronary syndrome more than 3 years after stent deployment.

## Case History

A 65-year-old man underwent coronary angiography for investigation of exertional angina. This conﬁrmed crit- ical lesions in the ﬁrst obtuse marginal branch of the circumﬂex artery (Fig. 1) with minor disease elsewhere and normal left ventriculography. Percutaneous coronary intervention to the culprit lesion was undertaken. The lesion was pre-dilated with $2.0\,\mathrm{mm}\times10\,\mathrm{mm}$  conventional angioplasty balloon to  $12\,\mathrm{atm.},$  followed by deployment of a $2.75\,\mathrm{mm}\times30\,\mathrm{mm}\;\mathrm{SE}$  result (Fig. 2).

Received 31 March 2007; received in revised form 29 April 2007; accepted 6 June 2007; available online 28 August 2007 The patient had an uneventful recovery and remained symptom free for almost 3.5 years, before re-presenting with a troponin positive acute coronary syndrome and electrocardiographic evidence of myocardial ischaemia in the lateral leads. Coronary angiography was repeated, revealing complete fracture of the previous SES near the midpoint. There was clear displacement of the stent segments from each other, resulting in a kink within the exposed artery (Fig. 3). The other coronary arteries remained unchanged. The stent fracture was treated with direct deployment of a  $3.0\,\mathrm{mm}\times13\,\mathrm{mm}$  SES to  $3.5\,\mathrm{mm}\times8\,\mathrm{mm}$  Powersail balloon to $12\,\mathrm{atm.},$  with an excellent ﬁnal result (Fig. 4). The patient remained well at 12-month follow-up, with a negative exercise treadmill test featuring normal exercise capacity.



## Discussion

Despite substantial clinical experience with drug-eluting stents in randomised trials and post-marketing reg- istry surveillance, coronary stent fracture has not been highlighted as a signiﬁcant clinical issue in the pub- lished literature. Nonetheless, awareness of this possible complication has recently begun to emerge with the publication of isolated case reports and small series. One such series suggested the prevalence of stent frac- ture may be as high as  $1.9\bar{\%},1$  with stent fracture occurringin10of530patientsundergoingclini- cally indicated repeat coronary angiography. All of these cases of stent fracture involved sirolimus-eluting
Thomer, O., Klaas, M., and Schr¨oder, W.

# Oblique Shock-Vortex Interaction

Breakdown of a slender vortex caused by an oblique (OSVI) shock is studied using numerical solutions of the Eu- ler equations for unsteady three-dimensional ﬂow. A Burgers vortex with a given circulation and axial velocity distribution is prescribed at the inﬂow boundary. The calculations show that like in incompressible ﬂows supersonic breakdown is primarily controlled by pressure forces. The shock is deformed into an ’s’-shaped part near the vortex core where the shock becomes normal. The results indicate that initiation of breakdown is more sensitive to variations in the axial velocity than in the circulation, and that the ﬂow structure is clearly time-dependent.

1. Introduction Three interaction modes are found, namely weak, moderate and strong interaction. For a weak shock-vortex inter- action the vortex passes through the shock without remarkably changing its shape. The moderate interaction mode yields a change of the shock contour and an inﬂuence on the vortex structure. The third mode, when breakdown occurs is called strong interaction.

The computed structure of the burst vortex part diﬀers from that obtained with normal shocks. It does not increase as much in the radial direction as that in the case of normal shock-vortex interaction. For moderate and strong interaction the shock is ’s’-shaped in the area of the vortex axis [2]. A small region of subsonic ﬂow is generated downstream of the deformed shock part. For the strong interaction mode a free stagnation point is formed inside the subsonic ﬂow pocket balanced by the recirculation region. The computations show that either a small axial or large azimuthal velocity component can trigger breakdown. The larger the pressure rise or the circulation or the lower the axial velocity component the more likely breakdown will occur. Stability is more dependent on the vortex wake than on the circulation.

2. Computational Setup and Boundary Conditions A wedge-like grid was generated. Although OSVI can be simulated using a rectangular grid in combination with the Rankine-Hugoniot equations a wedge aligned grid is more similar to the experimental setup. Due to the Prandtl-Meyer expansion caused by the convex curve at the aft end of the wedge stronger vortices (larger wake or higher circulation) are necessary to enforce breakdown. The breakdown is also dependent on the wedge geometry, the interchange between shock and expansion fan forces. Near the vortex axis the grid is clustered such that the vortex core () is resolved with approximately 21 grid points in the radial direction. The inﬂow conditions $-1<r<1$  are assumed to be described by a slender streamwise vortex centered at  $\scriptstyle{x=y=0}$  together with a radial distribution of the Gaussian axial velocity. Controlled by the sign of the parameter $\delta\in[0..1]$  inﬂow plane is modeled as a ‘wake-like’ proﬁle given by  $v_{z}(r)=v_{z_{\infty}}$   $(1-\delta\;e^{-\mu_{w}\cdot r^{2}})$ . The quantity ri = ±   $r_{i}=\pm\sqrt{1/(2\mu_{w})}$  describes the inﬂection point position for the Gaussian axial velocity proﬁle. The ratio of the azimuthal and the axial velocity components is deﬁned as the helix angle  $\tau=v_{\varphi}/v_{z}$  at the vortex core  $r=1$ . For a given azimuthal velocity distribution  $^{\prime}$  increases if either the circulation  $\Gamma_{0}$  and the wake parameter  $\delta$  are increased or the Mach number  $M\,a_{\infty}$  and the width of the proﬁle  $\mu_{w}$  are decreased. The inﬂow conditions are also used as initial conditions for the entire ﬂow ﬁeld.



For the OSVI a slender iso-energy Burgers vortex is used with  $v_{\varphi}(r)=\Gamma_{0}$  (1 − e ﬂow ﬁeld in the  $x$ - $\mathcal{Z}$ -middle plane of the computational domain for a strong interaction for  $M\,a_{\infty}=1.48$ ,  $\Gamma=2.69$ , , $\delta\,=\,0.245$   $\mu\,=\,0.4$  and  $\beta\,=\,11.5^{\circ}$  wedge angle. The Mach number distribution shows that the shock angle is particularly inclined in the vicinity of the vortex core. The originally straight oblique shock is distorted into an ’s’-shaped shock front near the vortex core. Downstream of the inclined shock a region of subsonic ﬂow occurs. With progressing time a small cone like shape emerges which is of the order of the vortex diameter. Compared to the normal shock case [1] this cap size is smaller. Since breakdown of the vortex is necessarily connected with the formation of a stagnation point on or near the axis, the increased shock angle is a must for the locally subsonic ﬂow downstream of the shock. As a second condition the pressure rise across the deformed part of the shock has to be suﬃcient to initiate breakdown. The streamlines inside the burst part of the vortex indicate recirculation and a stagnation point.



The parameter study for OSVI shows that the shock deformation in the vicinity of the vortex core depends primarily on the magnitude of the axial velocity deﬁcit in conjunction with the free-stream shock strength  $M\,a_{\infty}\sin\sigma$  and the wedge angle.Secondly, the vortex circulation seems to determine the pressure on the vortex axis.For a slender isentropic vortex the radial momentum equation and the energy equation are decoupled. In this case the inﬂuence of circulation is only linear on the axial momentum while the wake in of quadratic order. In this manner it is possible to catalogue the already explained OSVI modes:  $\beta_{A}\;<\;\beta_{c r i t}(M a_{A})\;\wedge\;\Upsilon\;<\;1$  or  $\Upsilon\ >\ 1$  leads to a weak shock vortex interaction mode.  $\Upsilon$  is the relation between forces upstream and downstream of the shock on the vortex axis. Both shock and vortex are only slightly inﬂuenced. The ﬂow ﬁeld remains supersonic everywhere.  stands for a moderate interaction mode. The shock resembles a strong solution in the vortex $\beta_{A}>\beta_{k r i t}(M a_{A})\wedge\Upsilon>1$  core vicinity. A small subsonic pocket accrues downstream of the shock. Elsewhere the ﬂow remains supersonic. The vortex is inﬂuenced without being destroyed. No stagnation point is observed. In the computational investigation of Nedungadi  $\&$  Lewis denotes $\beta_{A}>\beta_{k r i t}(M a_{A})\land\Upsilon<1$  a strong interaction. The shock has a similar form to that observed for moderate interaction. Additionally a larger subsonic region is formed and a stagnation point can be realized.

4. Concluding Remarks Breakdown of slender vortices in supersonic ﬂow caused by OSVI was investigated using numerical solutions of the Euler equations. The results show that a reduction of the axial velocity or an increase of the azimuthal velocity component enhance breakdown. While breakdown of vortices with uniform axial ﬂow is possible for normal shocks, a wake-like velocity proﬁle is required for oblique shocks.

5. References [1] O. THOMER, W. SCHR ¨ODER, E. KRAUSE, Normal and Oblique Shock-Vortex Interaction, Proceedings of International Conference RDAMM, Akademgorodok, Novosibirsk, Russia, June 24 - 29, vol. 6, no. 2, 2001.

[2] A. NEDUNGADI, M. LEWIS: Computational Study of the ﬂow-ﬁelds associated with oblique shock/vortex in- teractions, AIAA Journal, vol. 34, no. 12, pp. 2545-2553, 1996.

O. Thomer, M. Klaas, and W. Schr¨oder, AIA RWTH Aachen, W¨ullnerstr. zw. 5 u. 7, D-52062 Aachen.
Thomer, O., Klaas, M., and Schr¨oder, W.

# Oblique Shock-Vortex Interaction

Breakdown of a slender vortex caused by an oblique (OSVI) shock is studied using numerical solutions of the Eu- ler equations for unsteady three-dimensional ﬂow. A Burgers vortex with a given circulation and axial velocity distribution is prescribed at the inﬂow boundary. The calculations show that like in incompressible ﬂows supersonic breakdown is primarily controlled by pressure forces. The shock is deformed into an ’s’-shaped part near the vortex core where the shock becomes normal. The results indicate that initiation of breakdown is more sensitive to variations in the axial velocity than in the circulation, and that the ﬂow structure is clearly time-dependent.

1. Introduction Three interaction modes are found, namely weak, moderate and strong interaction. For a weak shock-vortex inter- action the vortex passes through the shock without remarkably changing its shape. The moderate interaction mode yields a change of the shock contour and an inﬂuence on the vortex structure. The third mode, when breakdown occurs is called strong interaction.

The computed structure of the burst vortex part diﬀers from that obtained with normal shocks. It does not increase as much in the radial direction as that in the case of normal shock-vortex interaction. For moderate and strong interaction the shock is ’s’-shaped in the area of the vortex axis [2]. A small region of subsonic ﬂow is generated downstream of the deformed shock part. For the strong interaction mode a free stagnation point is formed inside the subsonic ﬂow pocket balanced by the recirculation region. The computations show that either a small axial or large azimuthal velocity component can trigger breakdown. The larger the pressure rise or the circulation or the lower the axial velocity component the more likely breakdown will occur. Stability is more dependent on the vortex wake than on the circulation.

2. Computational Setup and Boundary Conditions A wedge-like grid was generated. Although OSVI can be simulated using a rectangular grid in combination with the Rankine-Hugoniot equations a wedge aligned grid is more similar to the experimental setup. Due to the Prandtl-Meyer expansion caused by the convex curve at the aft end of the wedge stronger vortices (larger wake or higher circulation) are necessary to enforce breakdown. The breakdown is also dependent on the wedge geometry, the interchange between shock and expansion fan forces. Near the vortex axis the grid is clustered such that the vortex core () is resolved with approximately 21 grid points in the radial direction. The inﬂow conditions $-1<r<1$  are assumed to be described by a slender streamwise vortex centered at  $\scriptstyle{x=y=0}$  together with a radial distribution of the Gaussian axial velocity. Controlled by the sign of the parameter $\delta\in[0..1]$  inﬂow plane is modeled as a ‘wake-like’ proﬁle given by  $v_{z}(r)=v_{z_{\infty}}$   $(1-\delta\;e^{-\mu_{w}\cdot r^{2}})$ . The quantity ri = ±   $r_{i}=\pm\sqrt{1/(2\mu_{w})}$  describes the inﬂection point position for the Gaussian axial velocity proﬁle. The ratio of the azimuthal and the axial velocity components is deﬁned as the helix angle  $\tau=v_{\varphi}/v_{z}$  at the vortex core  $r=1$ . For a given azimuthal velocity distribution  $^{\prime}$  increases if either the circulation  $\Gamma_{0}$  and the wake parameter  $\delta$  are increased or the Mach number  $M\,a_{\infty}$  and the width of the proﬁle  $\mu_{w}$  are decreased. The inﬂow conditions are also used as initial conditions for the entire ﬂow ﬁeld.



For the OSVI a slender iso-energy Burgers vortex is used with  $v_{\varphi}(r)=\Gamma_{0}$  (1 − e ﬂow ﬁeld in the  $x$ - $\mathcal{Z}$ -middle plane of the computational domain for a strong interaction for  $M\,a_{\infty}=1.48$ ,  $\Gamma=2.69$ , , $\delta\,=\,0.245$   $\mu\,=\,0.4$  and  $\beta\,=\,11.5^{\circ}$  wedge angle. The Mach number distribution shows that the shock angle is particularly inclined in the vicinity of the vortex core. The originally straight oblique shock is distorted into an ’s’-shaped shock front near the vortex core. Downstream of the inclined shock a region of subsonic ﬂow occurs. With progressing time a small cone like shape emerges which is of the order of the vortex diameter. Compared to the normal shock case [1] this cap size is smaller. Since breakdown of the vortex is necessarily connected with the formation of a stagnation point on or near the axis, the increased shock angle is a must for the locally subsonic ﬂow downstream of the shock. As a second condition the pressure rise across the deformed part of the shock has to be suﬃcient to initiate breakdown. The streamlines inside the burst part of the vortex indicate recirculation and a stagnation point.



The parameter study for OSVI shows that the shock deformation in the vicinity of the vortex core depends primarily on the magnitude of the axial velocity deﬁcit in conjunction with the free-stream shock strength  $M\,a_{\infty}\sin\sigma$  and the wedge angle.Secondly, the vortex circulation seems to determine the pressure on the vortex axis.For a slender isentropic vortex the radial momentum equation and the energy equation are decoupled. In this case the inﬂuence of circulation is only linear on the axial momentum while the wake in of quadratic order. In this manner it is possible to catalogue the already explained OSVI modes:  $\beta_{A}\;<\;\beta_{c r i t}(M a_{A})\;\wedge\;\Upsilon\;<\;1$  or  $\Upsilon\ >\ 1$  leads to a weak shock vortex interaction mode.  $\Upsilon$  is the relation between forces upstream and downstream of the shock on the vortex axis. Both shock and vortex are only slightly inﬂuenced. The ﬂow ﬁeld remains supersonic everywhere.  stands for a moderate interaction mode. The shock resembles a strong solution in the vortex $\beta_{A}>\beta_{k r i t}(M a_{A})\wedge\Upsilon>1$  core vicinity. A small subsonic pocket accrues downstream of the shock. Elsewhere the ﬂow remains supersonic. The vortex is inﬂuenced without being destroyed. No stagnation point is observed. In the computational investigation of Nedungadi  $\&$  Lewis denotes $\beta_{A}>\beta_{k r i t}(M a_{A})\land\Upsilon<1$  a strong interaction. The shock has a similar form to that observed for moderate interaction. Additionally a larger subsonic region is formed and a stagnation point can be realized.

4. Concluding Remarks Breakdown of slender vortices in supersonic ﬂow caused by OSVI was investigated using numerical solutions of the Euler equations. The results show that a reduction of the axial velocity or an increase of the azimuthal velocity component enhance breakdown. While breakdown of vortices with uniform axial ﬂow is possible for normal shocks, a wake-like velocity proﬁle is required for oblique shocks.

5. References [1] O. THOMER, W. SCHR ¨ODER, E. KRAUSE, Normal and Oblique Shock-Vortex Interaction, Proceedings of International Conference RDAMM, Akademgorodok, Novosibirsk, Russia, June 24 - 29, vol. 6, no. 2, 2001.

[2] A. NEDUNGADI, M. LEWIS: Computational Study of the ﬂow-ﬁelds associated with oblique shock/vortex in- teractions, AIAA Journal, vol. 34, no. 12, pp. 2545-2553, 1996.

O. Thomer, M. Klaas, and W. Schr¨oder, AIA RWTH Aachen, W¨ullnerstr. zw. 5 u. 7, D-52062 Aachen.
# Accelerometry-Based Prediction of Energy Expenditure in Preschoolers

Berit Steenbock Leibniz Institute for Prevention Research and Epidemiology – BIPS and University of Bremen Marvin N. Wright, Norman Wirsik, and Mirko Brandes Leibniz Institute for Prevention Research and Epidemiology – BIPS Purpose: Study purposes were to develop energy expenditure (EE) prediction models from raw accelerometer data and to investigate the performance of three different accelerometers on ﬁve different wear positions in preschoolers. Methods: Fourty-one children ( $54\%$  boys; 3–6.3 years) wore two Actigraph GT3X (left and right hip), three GENEActiv (right hip, left and right wrist), and one activPAL (right thigh) while completing a semi-structured protocol of 10 age-appropriate activities. Participants wore a portable indirect calorimeter to estimate EE. Utilized models to estimate EE included a linear model (LM), a mixed linear model (MLM), a random forest model (RF), and an artiﬁcial neural network model (ANN). For each accelerometer, model, and wear position, we assessed prediction accuracy via leave-one-out cross-validation and calculated the root-mean-squared-error (RMSE). Results: Mean RMSE ranged from  $2.56{-}2.76\,\mathrm{kJ/min}$  for the RF,  $2.72{-}3.08\,\mathrm{kJ/min}$  for the ANN,  $2.83{-}2.94\,\mathrm{kJ/min}$  for the LM, and  for the MLM. The GENEActive obtained mean RMSE of  $2.56\,\mathrm{kJ/min}$  (left and right wrist) and  $2.73\,\mathrm{kJ/min}$  (right $2.81–2.92\,\mathrm{kJ/min}$  hip). Predicting EE using the GT3X on the left and right hip obtained mean RMSE of 2.60 and  $2.74\,\mathrm{kJ/min}$ . The activPAL obtained a mean RMSE of  $2.76\,\mathrm{kJ/min}$ . Conclusion: These results demonstrate good prediction accuracy for recent accelerometers on different wear positions in preschoolers. The RF and ANN were equally accurate in EE prediction compared with (mixed) linear models. The RF seems to be a viable alternative to linear and ANN models for EE prediction in young children in a semi-structured setting.

 



Keywords: accelerometer, children, linear mixed model, machine learning, physical activity, validation It is generally agreed that regular physical activity (PA) is related to important health outcomes in children (e.g., cardiometa- bolic and psychosocial health; Knaeps et al., 2018; Reddon, Meyre, & Cairney, 2017; Shoup, Gattshall, Dandamudi, & Estabrooks, 2008; Skrede et al., 2017; Wafa et al., 2016). PA is deﬁned as any bodily movement produced by skeletal muscles that results in energy expenditure (EE) (Caspersen, Powell, & Christenson, 1985). In order to monitor children’s PA, analyze associations between PA and health outcomes, and evaluate the effectiveness of interventions promoting PA among children, valid measures of children’s PA and EE are needed (Lamonte & Ainsworth, 2001). In recent years, accelerometers have gained considerable popularity as an objective measure of sedentary behaviors, PA and other outcomes, such as EE. They detect accelerations of the body and enable an estimation of intensity, frequency, duration, and type of movement (Hills, Mokhtar, & Byrne, 2014; Skotte, Korshoj, Kristiansen, Hanisch, & Holtermann, 2014). Accelerometers have several advantages over traditional questionnaire-based measures of PA, including superior reliability and validity, and are increas- ingly being used in studies with very young children (Hills et al., 2014). However, traditional linear model equations developed for activity count-based data do not provide accurate estimates of EE in preschoolers (Janssen et al., 2013; Reilly et al., 2006).



Because the relationships between accelerometer output and EE differ in preschoolers compared with older children, prediction equations require development and validation in this speciﬁc age group (Butte et al., 2014). Considerable progress has been made in predicting EE for adults and older children (Jimmy, Seiler, & Maeder, 2013; Montoye, Begum, Henning, & Pfeiffer, 2017; Montoye, Mudd, Biswas, & Pfeiffer, 2015) whereas several methodological questions concerning the use of accelerometry in young children remain open. In their recently published review that provides age-speciﬁc practical considerations on accelerome- ter data collection (e.g., device placement) and processing criteria (e.g., epoch length, cut-points, and algorithms), Migueles and colleagues (2017) observed a lack of calibration and validation studies for preschoolers that address important processing criteria (such as EE algorithms for wrist- and hip-worn accelerometers). However, as studies included in the review were restricted to those applying the latest version of the Actigraph device (GT3X), no practical consideration about device selection in this age group could be drawn. Besides this, most of the research on EE prediction that has been done in preschoolers is limited by the use of direct observation as the criterion measurement and the assignment of ﬁxed metabolic equivalents (METs) to activities and accelerometer output (Davies et al., 2012; De Decker et al., 2013; Hagenbuchner, Cliff, Trost, Van Tuc, & Peoples, 2015). Additionally, the use of highly structured protocols under laboratory settings has been found to overestimate EE in children, which limits the transfer to free-living behaviors (Nilsson et al., 2008).

Recent studies in older children and adults show improve- ments in EE prediction using non-linear models (Mackintosh, Montoye, Pfeiffer, & McNarry, 2016; Montoye et al., 2017). Preliminary evidence demonstrates that machine learning models outperform simpliﬁed regression models in preschoolers (Chowdhury et al., 2018).



This study had three purposes related to accelerometer use in preschoolers: (1) to provide energy expenditure (EE) prediction models from raw accelerometry data established against indirect calorimetry, (2) to compare two linear and two machine learning models, and (3) to compare accuracy of different accelerometers placed on the hips, thigh, and wrists.

## Methods

### Study Participants

To recruit 3- to 6-year-old children from four daycare facilities (kindergartens) in Lower Saxony, we ﬁrst contacted the adminis- tration of the kindergartens. In the next step, we explained the aim and the procedures of the study to the administration as well as to parent representatives during a pre-arranged meeting in each kindergarten. Thereafter, study information material was distrib- uted to all parents. In the course of regular kindergarten morning circle time, we demonstrated the indirect calorimetry to all chil- dren. Children with written parental informed consent were allo- cated to a schedule. On the measurement day, we asked for verbal assent from each child and those refusing to take part in the study were excluded. The local ethics committee (University of Bremen, Bremen) approved the procedures of the study (ethics committee running title: accelerometry in preschoolers [ACCIPS]).

### Study Design

The measurements took place from October to November 2016 and were conducted at each respective kindergarten, using the available indoor and outdoor infrastructure. The children had to fast for at least two hours before measurement, but were allowed to drink water. The measurement was restricted to two children per morn- ing, measured one after the other, to avoid interference with lunchtime. A total of  $75~\mathrm{min}$  were allocated for the measurement protocol of one child. This included the assessment of anthropo- metrics, handedness, mounting and demounting the devices, mea- surement of resting metabolic rate (RMR), performance of up to nine in- and outdoor activities, changing rooms within the kinder- garten, and dressing and undressing for outdoor activities (Table 1). We measured RMR during 10 min of supine rest in a quiet, dimly lit room. The child lay on a mattress and was provided with a blanket. A short learning story was shown on a tablet to keep the child calm for the measurement of RMR. We decided on a measurement period of 10 minutes as Borges et al. (2016) showed that this is an appropriate length of time to achieve steady state conditions and delineate an optimum abbreviated period to estimate RMR by indirect calorimetry.

### Activities

The child was free to choose up to three friends to play with. Five activities were predetermined and ranged from light (e.g., drawing) to vigorous intensities (e.g., jogging). An addi- tional four activities to be chosen independently with respect to the resources of the daycare facility and the child’s individual preferences were offered to each child. Examples are given in Table 1. Starting with the ﬁrst predetermined activity ‘drawing’, the child was asked to sit down for ﬁve minutes at a table in a group room and to choose one out of ﬁve coloring pages with different motifs to crayon. All subsequent activities were per- formed continuously for at least three minutes. Additionally, the child could choose two indoor physical activities (e.g., playing with toy cars, toy blocks, hide and seek, playing with dolls). For the performance of the outdoor activities, a well-deﬁned and plain area was chosen. On the outdoors, the child started with a predetermined activity, which was ‘tag’. Afterwards, the child chose two other outdoor activities, depending on the equipment of each of the four local daycare facilities (e.g., swing, tricycle, monkey bars, or scooter). Finally, the child performed the three activities scheduled last, which were walking at normal speed, walking fast, and jogging. During these activities, the researcher accompanied the child to ensure continuous walking and jogging, while the child set the pace. For fast walking, the researcher encouraged the child by telling an imaginary journey of a holiday trip, including catching the tram, train, and plane to get to a holiday location. If the child changed to jogging during the fast walking task, the researcher immediately slowed the child down to fast walking. If the child refused to perform an activity continu- ously due to motivational reasons, the activity was stopped and the child was asked to continue with the next activity. The protocol was completely aborted if the child was exhausted or refused to go on any more.



### Accelerometers

The children were equipped with six accelerometers of three differ- ent brands. One accelerometer was placed at the left and the right wrist (GENEActiv, ActivInsight Ltd, Kimbolton, UK), one acceler- ometer at the left hip (GT3X Actigraph, Pensacola, Florida, USA), two accelerometers at the right hip (GENEActiv, GT3X), and one accelerometer at the right thigh (activPAL, PAL Technologies Ltd, Glasgow, Scotland, UK). Accelerometers at the wrist were ﬁrmly mounted with  $\%$  inch Tyvek security bracelets, which were used instead of the standard bracelets provided by the manufacturer. An elastic belt was used to ﬁx the hip-worn monitors close to the lateral hip bone, and an adhesive pad was used to ﬁx the activPAL to the skin on the front of the right thigh. All accelerometers were operated with the highest possible resolution, which was  $100\,\mathrm{~Hz~}$  in the GENEActiv and the GT3X, and  $20\,\mathrm{~Hz~}$  in the activPAL. Before each measurement we initialized and synchronized the acceler- ometers with the computer running the indirect calorimetry. After completion of the protocol we downloaded accelerometer data from all devices using dedicated software from the manufacturers.

### Physiological Measures

A portable, open-circuit indirect calorimetry system (MetaMax3b, Cortex Biophysics, Leipzig, Germany) was used as the criterion measure. The MetaMax3b has proven reliability in a study with adolescents but was found to slightly overestimate  $\mathrm{VO}_{2}$  during moderate and vigorous exercise (Macfarlane & Wong, 2012). It is highly comparable to other common devices (Brandes, Klein, Ginsel, & Heitmann, 2015). The system was mounted to the children using a pediatric harness. In very small children, addi- tional adhesive tape was used to adjust the harness to the child, so that the system did not interfere with the activities. A facemask was secured over the child´s nose and mouth with an adjustable nylon harness. A bidirectional turbine, inserted to the facemask, mea- sured the volume of inspired and expired air. A sample tube, connected to the turbine, retrieved expired air samples breath-by- breath. Air samples were analyzed for oxygen uptake and carbon dioxide production within the sensor unit of the system. Stable and dry weather conditions enabled sound outdoor measurements. Data were transferred via telemetry to a laptop and available in real-time (MetaSoft 3, Cortex Biophysics, Leipzig, Germany). The laptop operator used the marker function of the software to identify the beginning and the end of each activity. Each morning, prior to the ﬁrst measurement, we calibrated the indirect calorimetry system according to the manufacturer recommended procedure.

Data Reduction and Modeling Oxygen consumption and carbon dioxide production were measured continuously and converted to absolute  $(\mathrm{kJ/min})$  and relative  $\mathrm{(J/min/kg)}$  EE using the equations by Weir (1949). RMR



Note. Data are displayed as  $M\pm S D$  (min, max).

was calculated as the minimum of a rolling one-minute mean during supine rest. Metabolic equivalents (METs) were calcu- lated by dividing the relative oxygen consumption (expressed in  $\mathrm{ml/min/kg)}$  by 3.5.

As done by Montoye et al. (2017), for each individual and device, the following 10 summary statistics were calculated in 30-s non-overlapping windows for each axis  $(\mathrm{X},\mathrm{Y},\mathrm{Z})$  of the raw acceleration data—mean, standard deviation, minimum, maximum, lag one autocorrelation, percentiles  $(10\%,25\%,50\%,75\%,90\%)-$ — resulting in 30 summary statistics for each time window. No ﬁltering of the raw accelerometer data was conducted.

Predictive models were created separately for each acceler- ometer and placement, resulting in 24  $(4\times6)$  models developed and tested. We built four predictive models: (1) a linear regression model, (2) a linear mixed model, (3) a random forest, and (4) an artiﬁcial neural network model. Linear regression models (LM) and linear mixed models (MLM) were created separately for the outcome variables absolute EE (absEE), relative EE (relEE), and METs and included the above mentioned 30 summary statistics as independent variables. The MLM accounts for repeated measure- ments to compute EE prediction.

Random forests (RF) are ensembles of classiﬁcation or regres- sion trees. Each tree is grown on a bootstrap sample of the data to create an ensemble of diverse trees, modelling different aspects of the data. Further randomization is introduced by drawing a random subset of variables at every split. We used regression RFs to predict EE (METs, absEE, or relEE) with raw acceleration data, based on the 30 summary statistics.

Artiﬁcial neural networks (ANNs), inspired by biological neural networks, are widely used to model complex relationships between inputs and outputs. In contrast to the LM and MLM, they do not assume a speciﬁc type of relationship between the outcome and covariates and allow for non-linear dependencies. In this study, ANNs were used to approximate the functional relationship between the response (METs, absEE, or relEE) and the 30 summary statistics as covariates.

All analyses were performed in R (R Core Team, 2017) with the add-on packages ‘lme4’ (Bates, Mächler, Bolker, & Walker, 2015) for MLM, ‘ranger’ (Wright & Ziegler, 2017) for RF, and ‘nnet’ (Venables & Ripley, 2002) for ANN. All model ﬁtting, cross validation, parameter tuning, and evaluation was performed with the ‘mlr’ (Bischl et al., 2016) package. In the RF, we grew 500 trees and tuned the parameter ‘mtry’ using model-based optimization (Bischl et al., 2017). For the ANN we chose a single hidden layer with 15 units and tuned the regularization parameter ‘decay’ to avoid overﬁtting. Code for using the developed models can be found at the following link: https://github.com/bips-hb/EE_prediction.



### Statistical Analysis

Separately for each of the models, accelerometers, and positions, we calculated the root-mean-squared-error (RMSE) to test EE prediction accuracy. We tested all models using a leave-one-out cross-validation.

## Results

We had written informed consent from parents of 62 children. Of these, 41 children completed the protocol. All of the partici- pating children were Caucasians and were able to speak and understand German. Participant characteristics can be found in Table 2.

Reasons for non-participation were withdrawals of the consent on the measurement day (11), illness (5), not being fasted for at least two hours (2), withdrawn after the second activity (1), and missing data due to calorimeter failure (2). Children completed the protocol with a median of eight out of nine possible activities. Mean EE per activity ranged from  $4.2\pm0.9\,\mathrm{kJ/min}$  to  $16.3\pm4.6\,\mathrm{kJ/min}$  (Table 3). Forty  $(98\%)$  children had valid data for the GENEActiv on the left wrist and for the GT3X on the right hip. Thirty-nine  $(95\%)$  children had valid data for the GENEActiv on the right wrist and for the GT3X on the left hip. Thirty-eight  $(93\%)$  children had valid data for the GENEActiv on the right hip and for the activPAL. Predicted METs from the four different models (GENEActiv, left wrist) and METs derived from the indirect calorimeter of one child passing the protocol are exemplarily shown in Figure 1.

Boxplots for RMSE for all accelerometers, locations, and models are shown in Figures 2–4 for absEE, relEE, and METs, respectively.

Mean RMSE ranged from  $2.56{-}2.76\;\mathrm{kJ/min}$  for the RF, from  for the ANN, from  $2.83{-}2.94\,\mathrm{kJ/min}$  $2.72{-}3.08\,\mathrm{kJ/min}$  for the LM, and from  $2.81{-}2.92\;\mathrm{kJ/min}$  for the MLM (Table 4). A comparison of the four models shows that the RF revealed slightly lower RMSE for absEE, relEE and METs than the other three models. GENEActive devices obtained mean RMSE of  $2.56~\mathrm{kJ/min}$  (left and right wrist) and  $2.73\,\mathrm{kJ/min}$  (right hip). Predicting EE using the GT3X on the left and right hip obtained mean RMSE of 2.60 and  $2.74~\mathrm{kJ/min}.$ , respectively. Use of the thigh worn activPAL pro- vided a mean RMSE of  $2.76\,\mathrm{kJ/min}$ . Looking at wear positions, EE prediction accuracy for devices worn at the hip obtained mean RMSE of  $2.60{-}2.74\,\mathrm{\kJ/min}$ . Placement at the wrists obtained slightly lower mean RMSE  $2.56~\mathrm{kJ/min})$ , and placement at the thigh slightly higher mean RMSE  $(2.76~\mathrm{kJ/min})$  (Table 4).



## Discussion

The study’s purpose was to develop and to provide EE prediction models from raw accelerometry data, to compare two linear and two machine learning models, and to compare EE prediction accuracy of different accelerometers placed on the hips, thigh, and wrists. To our knowledge, this is the ﬁrst study that provides a calibration and comparative validation of multiple accelerometers in preschool children aged three to six years against indirect calorimetry as the criterion measurement.



For adults, similar EE prediction accuracy has been reported for linear models vs. an ANN model using different accelerometers (Montoye et al., 2017). Only for wrist-worn devices, Montoye et al. (2017) report a signiﬁcant improvement of the ANN over linear models. In children, there is preliminary evidence that the applica- tion of machine learning improves EE prediction accuracy com- pared to linear regression algorithms (Chowdhury et al., 2018; Mackintosh et al., 2016). In our study, machine learning models and (mixed) linear models were equally accurate in EE prediction irrespective of the device and wear position. This could be due to the fact, that our models all included the same summary statistics as independent variables whereas in the simpliﬁed approach by Chowdhury et al. (2018) only three features, including mean  $(x)$ , mean (y), mean (z), were extracted from each accelerometer window. Thus our data shows preliminary evidence that applica- tion of an RF or ANN model leads to only minor improvements in EE prediction compared to (mixed) linear models if the models are built upon the same summary statistics as independent vari- ables. We can therefore recommend using linear as well as non-linear models for EE prediction on raw accelerometry data in studies that aim at assessing young children’s EE under free-living conditions. Furthermore, comparison of mean RMSE did not reveal meaningful differences between the three different devices. All of them can therefore be recommended for use in young children.



Accelerometers are commonly worn on a waist belt, aligned with the right anterior axillary line for the entire day up to seven days to estimate habitual PA (Hills et al., 2014). To date, most research has used the hip-worn GT3X, which has been calibrated and validated in a wide range of populations (Borghese et al., 2017; Evenson et al., 2015; Johansson, Larisch, Marcus, & Hagstromer, 2016). Compliance among young children has been shown to be better when using wrist-worn accelerometers compared to hip- worn devices (Fairclough et al., 2016). Therefore, our main aim was to test whether EE prediction of wrist-, hip- and thigh-worn devices is equally accurate for age-appropriate activities in pre- schoolers. Our results show good accuracy for all wear positions. This is in line with the ﬁndings of another study with children in which accelerometers mounted on various anatomical positions demonstrated equivalency in the accuracy to predict EE in a semi- structured setting (Mackintosh et al., 2016).

In our study, we further observed only small differences regarding the predictive accuracy between the dominant and non-dominant wrist. This also is in line with the ﬁndings of another study that examined the classiﬁcation accuracy of the GENEActiv with a cut-point based approach for the assessment of PA intensi- ties in preschoolers (Roscoe, James, & Duncan, 2017) and may help to inform choice of wrist placement when using acceler- ometers in ﬁeld-based research. We assume good accuracy in predicting EE should the accelerometer either be placed on the left or right wrist.

There are several strengths of our study which are in line with best practice recommendations for calibration studies (Welk, 2005). We applied a semi-structured nature of the activity protocol, numerous accelerometer positions, and a high-quality criterion measure for EE. Our sample size of 41 children is in line with recent calibration and validation studies and the included activities enabled their differentiation into light- to vigorous intensity activi- ties quite well, with the lowest activity-speciﬁc metabolic equiva- lents (AME) being observed in drawing [mean  $(95\%$  CI): 1.89 (1.7–2.1)], a moderate AME in playing with cars [3.1 (2.5–3.6)] and walking [3.6 (3.3–4.0)], and the most vigorous AME in jogging [7.7 (6.7–8.7)] (Brandes, Steenbock, & Wirsik, 2018). The generalizability of our results is indeed limited by the homog- enous sample of apparently healthy Caucasian children. Therefore, our ﬁndings cannot be generalized to other ethnicities or to children with a chronic condition or disease. A further limitation of the study is that not all children completed the ﬁve predetermined activities that stretch across all activity intensities (drawing, catching, walk- ing at regular speed, walking fast, and jogging). This was due to the young age and motivational reasons in our target group.



## Conclusion

These ﬁndings provide preliminary evidence that recent acceler- ometers mounted on ﬁve different anatomical positions demon- strate equivalency in the accuracy to predict EE from raw accelerometer data. Our models are publicly available and can be used for studies assessing EE in preschoolers under free-living conditions. The RF, ANN, and (mixed) linear models were equally accurate in predicting EE irrespective of the device and wear position. We therefore recommend utilizing linear as well as non-linear models for the estimation of EE in preschoolers. We recommend a further investigation of RF models for EE prediction from raw accelerometer data in young children as well as in other populations, which seems to be a viable alternative to linear and ANN models.

Note. LM $=$ linear model; MLM $=$ mixed linear model;  $\mathrm{RF}=1$ random forest; ANN $=$ artiﬁcial neural network; RMSE $=$ root-mean-square error.

### Acknowledgments

The study was supported by an internal innovations grant of the Leibniz Institute for Prevention Research and Epidemiology (BIPS). We thank Johanna Sophie Lubasch und Marlena Böning for their assistance during data collection and all daycare facilities, daycare staff, parents and children for their support and participation in the study. The authors declare that they have no conﬂicts of interest.

## References

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01 Bischl, B., Lang, M., Kotthoff, L., Schiffner, J., Richter, J., Studerus, E., : : : Jones, Z.M. (2016). mlr: Machine Learning in R. The Journal of Machine Learning Research, 17(170), 1–5.



Bischl, B., Richter, J., Bossek, J., Horn, D., Thomas, J., & Lang, M. (2017). mlrMBO: A Modular Framework for Model-Based Optimi- zation of expensive black-box functions. Retrieved from http://arxiv. org/abs/1703.03373 Borges, J.H., Langer, R.D., Cirolini, V.X., Pascoa, M.A., Guerra-Junior, G., & Goncalves, E.M. (2016). Minimum time to achieve the steady state and optimum abbreviated period to estimate the resting energy expenditure by indirect calorimetry in healthy young adults. Nutrition in Clinical Practice, 31(3), 349–354. PubMed ID: 26888859 Borghese, M.M., Tremblay, M.S., LeBlanc, A.G., Leduc, G., Boyer, C., & Chaput, J.P. (2017). Comparison of ActiGraph GT3X+ and actical accelerometer data in 9–11-year-old Canadian children. Journal of Sports Sciences, 35(6), 517–524. PubMed ID: 27103499 doi:10. 1080/02640414.2016.1175653 Brandes, M., Klein, H., Ginsel, S., & Heitmann, A. (2015). Comparability of three mobile respiratory gas analyzers. Sportwissenschaft, 45(4), 168–172. doi:10.1007/s12662-015-0368- $\mathrm{X}$  Brandes, M., Steenbock, B., & Wirsik, N. (2018). Energy cost of common physical activities in preschoolers. Journal of Physical Activity & Health, 15(4), 233–238. PubMed ID: 29202634 doi:10.1123/jpah. 2017-0348 Butte, N.F., Wong, W.W., Lee, J.S., Adolph, A.L., Puyau, M.R., & Zakeri, I.F. (2014). Prediction of energy expenditure and physical activity in preschoolers. Medicine & Science in Sports & Exercise, 46(6), 1216– 1226. PubMed ID: 24195866 doi:10.1249/mss.0000000000000209 Caspersen, C.J., Powell, K.E., & Christenson, G.M. (1985). Physical activity, exercise, and physical ﬁtness: Deﬁnitions and distinctions for health-related research. Public Health Rep, 100(2), 126–131. PubMed ID: 3920711 Chowdhury, A.K., Tjondronegoro, D., Zhang, J., Hagenbuchner, M., Cliff, D., & Trost, S.G. (2018). Deep learning for energy expen- diture prediction in pre-school children. Paper presented at the IEEE Conference on Biomedical and Health Informatics, Las Vegas, NA.

      

     

    

   

  

 



Davies, G., Reilly, J., McGowan, A., Dall, P., Granat, M., & Paton, J. (2012). Validity, practical utility, and reliability of the activPAL in preschool children. Medicine & Science in Sports & Exercise, 44(4), 761–768. PubMed ID: 21983077 De Decker, E., De Craemer, M., Santos-Lozano, A., Van Cauwenberghe, E., De Bourdeaudhuij, I., & Cardon, G. (2013). Validity of the ActivPAL and the ActiGraph monitors in preschoolers. Medicine & Science in Sports & Exercise, 45(10), 2002–2011. PubMed ID: 23524516 doi:10.1249/MSS.0b013e318292c575 Evenson, K.R., Wen, F., Herring, A.H., Di, C., LaMonte, M.J., Tinker, L.F.,: : :Buchner, D.M. (2015). Calibrating physical activity intensity for hip-worn accelerometry in women age 60 to 91 years: The Women’s Health Initiative OPACH Calibration Study. Preven- tive Medicine Reports, 2, 750–756. PubMed ID: 26527313 doi:10. 1016/j.pmedr.2015.08.021 Fairclough, S.J., Noonan, R., Rowlands, A.V., Van Hees, V., Knowles, Z., & Boddy, L.M. (2016). Wear compliance and activity in children wearing wrist- and hip-mounted accelerometers. Medicine & Science in Sports & Exercise, 48(2), 245–253. PubMed ID: 26375253 doi:10. 1249/mss.0000000000000771 Hagenbuchner, M., Cliff, D.P., Trost, S.G., Van Tuc, N., & Peoples, G.E. (2015). Prediction of activity type in preschool children using machine learning techniques. Journal of Science and Medicine in Sport, 18(4), 426–431. PubMed ID: 25088983 Hills, A.P., Mokhtar, N., & Byrne, N.M. (2014). Assessment of physical activity and energy expenditure: An overview of objective measures. Frontiers in Nutrition, 1, 5. PubMed ID: 25988109 doi:10.3389/fnut. 2014.00005 Janssen, X., Cliff, D.P., Reilly, J.J., Hinkley, T., Jones, R.A., Batterham, M., : : : Okely, A.D. (2013). Predictive validity and classiﬁcation accuracy of ActiGraph energy expenditure equations and cut-points in young children. PLoS One, 8(11), e79124. PubMed ID: 24244433 doi:10.1371/journal.pone.0079124 Jimmy, G., Seiler, R., & Maeder, U. (2013). Development and validation of energy expenditure prediction models based on GT3X accelerom- eter data in 5- to 9-year-old children. Journal of Physical Activity & Health, 10(7), 1057–1067. PubMed ID: 23136371 Johansson, E., Larisch, L.M., Marcus, C., & Hagstromer, M. (2016). Calibration and validation of a wrist- and hip-worn actigraph accel- erometer in 4-year-old children. PLoS One, 11(9), e0162436. PubMed ID: 27617962 doi:10.1371/journal.pone.0162436 Knaeps, S., Bourgois, J.G., Charlier, R., Mertens, E., Lefevre, J., & Wijndaele, K. (2018). Ten-year change in sedentary behaviour, moderate-to-vigorous physical activity, cardiorespiratory ﬁtness and cardiometabolic risk: Independent associations and mediation analysis. British Journal of Sports Medicine, 52(16), 1063–1068. PubMed ID: 27491779 doi:10.1136/bjsports-2016-096083 Lamonte, M.J., & Ainsworth, B.E. (2001). Quantifying energy expendi- ture and physical activity in the context of dose response. Medicine & Science in Sports & Exercise, 33(Suppl. 6), S370–S378. discussion S419–S320. PubMed ID: 11427762 Macfarlane, D.J., & Wong, P. (2012). Validity, reliability and stability of the portable cortex metamax 3B gas analysis system. European Journal of Applied Physiology, 112(7), 2539–2547. PubMed ID: 22075643 doi:10.1007/s00421-011-2230-7 Mackintosh, K.A., Montoye, A.H., Pfeiffer, K.A., & McNarry, M.A. (2016). Investigating optimal accelerometer placement for energy expenditure prediction in children using a machine learning approach. Physiological Measurement, 37(10), 1728–1740. PubMed ID: 27653339 doi:10.1088/0967-3334/37/10/1728 Migueles, J.H., Cadenas-Sanchez, C., Ekelund, U., Delisle Nystrom, C., Mora-Gonzalez, J., Lof, M., : : : Ortega, F.B. (2017). Accelerometer data collection and processing criteria to assess physical activity and other outcomes: A systematic review and practical considerations. Sports Medicine, 47(9), 1821–1845. PubMed ID: 28303543 doi:10. 1007/s40279-017-0716-0 Montoye, A.H., Begum, M., Henning, Z., & Pfeiffer, K.A. (2017). Comparison of linear and non-linear models for predicting energy expenditure from raw accelerometer data. Physiological Measure- ment, 38(2), 343–357. PubMed ID: 28107205 doi:10.1088/1361- 6579/38/2/343 Montoye, A.H., Mudd, L.M., Biswas, S., & Pfeiffer, K.A. (2015). Energy expenditure prediction using raw accelerometer data in simulated free living. Medicine & Science in Sports & Exercise, 47(8), 1735–1746. PubMed ID: 25494392 doi:10.1249/mss.0000000000000597 Nilsson, A., Brage, S., Riddoch, C., Anderssen, S.A., Sardinha, L.B., Wedderkopp, N., : : : Ekelund, U. (2008). Comparison of equations for predicting energy expenditure from accelerometer counts in children. Scandinavian Journal of Medicine & Science in Sports, 18(5), 643–650. PubMed ID: 18208433 doi:10.1111/j.1600-0838. 2007.00694.x R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/ Reddon, H., Meyre, D., & Cairney, J. (2017). Physical activity and global self-worth in a longitudinal study of children. Medicine & Science in Sports & Exercise, 49(8), 1606–1613. PubMed ID: 28319588 doi:10. 1249/mss.0000000000001275 Reilly, J.J., Kelly, L.A., Montgomery, C., Jackson, D.M., Slater, C., Grant, S., & Paton, J.Y. (2006). Validation of Actigraph accelerometer estimates of total energy expenditure in young children. International Journal of Pediatric Obesity, 1(3), 161–167. PubMed ID: 17899634 Roscoe, C.M.P., James, R.S., & Duncan, M.J. (2017). Calibration of GENEActiv accelerometer wrist cut-points for the assessment of physical activity intensity of preschool aged children. European Journal of Pediatrics, 176(8), 1093–1098. PubMed ID: 28674825 doi:10.1007/s00431-017-2948-2 Shoup, J.A., Gattshall, M., Dandamudi, P., & Estabrooks, P. (2008). Physical activity, quality of life, and weight status in overweight children. Quality of Life Research, 17(3), 407–412. PubMed ID: 18293100 doi:10.1007/s11136-008-9312-y Skotte, J., Korshoj, M., Kristiansen, J., Hanisch, C., & Holtermann, A. (2014). Detection of physical activity types using triaxial acceler- ometers. Journal of Physical Activity & Health, 11(1), 76–84. PubMed ID: 23249722 doi:10.1123/jpah.2011-0347 Skrede, T., Stavnsbo, M., Aadland, E., Aadland, K.N., Anderssen, S.A., Resaland, G.K., & Ekelund, U. (2017). Moderate-to-vigorous physical activity, but not sedentary time, predicts changes in cardiometabolic risk factors in 10-y-old children: The active smarter kids study. The American Journal of Clinical Nutrition, 105(6), 1391–1398. PubMed ID: 28381476 doi:10.3945/ajcn.116. 150540 Venables, W.N., & Ripley, B.D. (2002). Modern Applied Statistics with S (4th ed.). New York, NY: Springer.

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 



Wafa, S.W., Shahril, M.R., Ahmad, A.B., Zainuddin, L.R., Ismail, K.F., Aung, M.M., & Mohd Yusoff, N.A. (2016). Association between physical activity and health-related quality of life in children: A cross- sectional study. Health and Quality of Life Outcomes, 14, 71. PubMed ID: 27146199 doi:10.1186/s12955-016-0474-y Weir, J.B. (1949). New methods for calculating metabolic rate with special reference to protein metabolism. The Journal of Physiology, 109(1–2), 1–9. PubMed ID: 15394301 Welk, G.J. (2005). Principles of design and analyses for the calibration of accelerometry-based activity monitors. Medicine & Science in Sports & Exercise, 37(Suppl. 11), S501–S511. PubMed ID: 16294113 Wright, M.N., & Ziegler, A. (2017). Ranger: A fast implementation of random forests for high dimensional data in  $C{++}$  and R. Journal of Statistical Software, 77(1), 1–17. doi:10.18637/jss.v077.i01

  

 


# Accelerometry-Based Prediction of Energy Expenditure in Preschoolers

Berit Steenbock Leibniz Institute for Prevention Research and Epidemiology – BIPS and University of Bremen Marvin N. Wright, Norman Wirsik, and Mirko Brandes Leibniz Institute for Prevention Research and Epidemiology – BIPS Purpose: Study purposes were to develop energy expenditure (EE) prediction models from raw accelerometer data and to investigate the performance of three different accelerometers on ﬁve different wear positions in preschoolers. Methods: Fourty-one children ( $54\%$  boys; 3–6.3 years) wore two Actigraph GT3X (left and right hip), three GENEActiv (right hip, left and right wrist), and one activPAL (right thigh) while completing a semi-structured protocol of 10 age-appropriate activities. Participants wore a portable indirect calorimeter to estimate EE. Utilized models to estimate EE included a linear model (LM), a mixed linear model (MLM), a random forest model (RF), and an artiﬁcial neural network model (ANN). For each accelerometer, model, and wear position, we assessed prediction accuracy via leave-one-out cross-validation and calculated the root-mean-squared-error (RMSE). Results: Mean RMSE ranged from  $2.56{-}2.76\,\mathrm{kJ/min}$  for the RF,  $2.72{-}3.08\,\mathrm{kJ/min}$  for the ANN,  $2.83{-}2.94\,\mathrm{kJ/min}$  for the LM, and  for the MLM. The GENEActive obtained mean RMSE of  $2.56\,\mathrm{kJ/min}$  (left and right wrist) and  $2.73\,\mathrm{kJ/min}$  (right $2.81–2.92\,\mathrm{kJ/min}$  hip). Predicting EE using the GT3X on the left and right hip obtained mean RMSE of 2.60 and  $2.74\,\mathrm{kJ/min}$ . The activPAL obtained a mean RMSE of  $2.76\,\mathrm{kJ/min}$ . Conclusion: These results demonstrate good prediction accuracy for recent accelerometers on different wear positions in preschoolers. The RF and ANN were equally accurate in EE prediction compared with (mixed) linear models. The RF seems to be a viable alternative to linear and ANN models for EE prediction in young children in a semi-structured setting.

 



Keywords: accelerometer, children, linear mixed model, machine learning, physical activity, validation It is generally agreed that regular physical activity (PA) is related to important health outcomes in children (e.g., cardiometa- bolic and psychosocial health; Knaeps et al., 2018; Reddon, Meyre, & Cairney, 2017; Shoup, Gattshall, Dandamudi, & Estabrooks, 2008; Skrede et al., 2017; Wafa et al., 2016). PA is deﬁned as any bodily movement produced by skeletal muscles that results in energy expenditure (EE) (Caspersen, Powell, & Christenson, 1985). In order to monitor children’s PA, analyze associations between PA and health outcomes, and evaluate the effectiveness of interventions promoting PA among children, valid measures of children’s PA and EE are needed (Lamonte & Ainsworth, 2001). In recent years, accelerometers have gained considerable popularity as an objective measure of sedentary behaviors, PA and other outcomes, such as EE. They detect accelerations of the body and enable an estimation of intensity, frequency, duration, and type of movement (Hills, Mokhtar, & Byrne, 2014; Skotte, Korshoj, Kristiansen, Hanisch, & Holtermann, 2014). Accelerometers have several advantages over traditional questionnaire-based measures of PA, including superior reliability and validity, and are increas- ingly being used in studies with very young children (Hills et al., 2014). However, traditional linear model equations developed for activity count-based data do not provide accurate estimates of EE in preschoolers (Janssen et al., 2013; Reilly et al., 2006).



Because the relationships between accelerometer output and EE differ in preschoolers compared with older children, prediction equations require development and validation in this speciﬁc age group (Butte et al., 2014). Considerable progress has been made in predicting EE for adults and older children (Jimmy, Seiler, & Maeder, 2013; Montoye, Begum, Henning, & Pfeiffer, 2017; Montoye, Mudd, Biswas, & Pfeiffer, 2015) whereas several methodological questions concerning the use of accelerometry in young children remain open. In their recently published review that provides age-speciﬁc practical considerations on accelerome- ter data collection (e.g., device placement) and processing criteria (e.g., epoch length, cut-points, and algorithms), Migueles and colleagues (2017) observed a lack of calibration and validation studies for preschoolers that address important processing criteria (such as EE algorithms for wrist- and hip-worn accelerometers). However, as studies included in the review were restricted to those applying the latest version of the Actigraph device (GT3X), no practical consideration about device selection in this age group could be drawn. Besides this, most of the research on EE prediction that has been done in preschoolers is limited by the use of direct observation as the criterion measurement and the assignment of ﬁxed metabolic equivalents (METs) to activities and accelerometer output (Davies et al., 2012; De Decker et al., 2013; Hagenbuchner, Cliff, Trost, Van Tuc, & Peoples, 2015). Additionally, the use of highly structured protocols under laboratory settings has been found to overestimate EE in children, which limits the transfer to free-living behaviors (Nilsson et al., 2008).

Recent studies in older children and adults show improve- ments in EE prediction using non-linear models (Mackintosh, Montoye, Pfeiffer, & McNarry, 2016; Montoye et al., 2017). Preliminary evidence demonstrates that machine learning models outperform simpliﬁed regression models in preschoolers (Chowdhury et al., 2018).



This study had three purposes related to accelerometer use in preschoolers: (1) to provide energy expenditure (EE) prediction models from raw accelerometry data established against indirect calorimetry, (2) to compare two linear and two machine learning models, and (3) to compare accuracy of different accelerometers placed on the hips, thigh, and wrists.

## Methods

### Study Participants

To recruit 3- to 6-year-old children from four daycare facilities (kindergartens) in Lower Saxony, we ﬁrst contacted the adminis- tration of the kindergartens. In the next step, we explained the aim and the procedures of the study to the administration as well as to parent representatives during a pre-arranged meeting in each kindergarten. Thereafter, study information material was distrib- uted to all parents. In the course of regular kindergarten morning circle time, we demonstrated the indirect calorimetry to all chil- dren. Children with written parental informed consent were allo- cated to a schedule. On the measurement day, we asked for verbal assent from each child and those refusing to take part in the study were excluded. The local ethics committee (University of Bremen, Bremen) approved the procedures of the study (ethics committee running title: accelerometry in preschoolers [ACCIPS]).

### Study Design

The measurements took place from October to November 2016 and were conducted at each respective kindergarten, using the available indoor and outdoor infrastructure. The children had to fast for at least two hours before measurement, but were allowed to drink water. The measurement was restricted to two children per morn- ing, measured one after the other, to avoid interference with lunchtime. A total of  $75~\mathrm{min}$  were allocated for the measurement protocol of one child. This included the assessment of anthropo- metrics, handedness, mounting and demounting the devices, mea- surement of resting metabolic rate (RMR), performance of up to nine in- and outdoor activities, changing rooms within the kinder- garten, and dressing and undressing for outdoor activities (Table 1). We measured RMR during 10 min of supine rest in a quiet, dimly lit room. The child lay on a mattress and was provided with a blanket. A short learning story was shown on a tablet to keep the child calm for the measurement of RMR. We decided on a measurement period of 10 minutes as Borges et al. (2016) showed that this is an appropriate length of time to achieve steady state conditions and delineate an optimum abbreviated period to estimate RMR by indirect calorimetry.

### Activities

The child was free to choose up to three friends to play with. Five activities were predetermined and ranged from light (e.g., drawing) to vigorous intensities (e.g., jogging). An addi- tional four activities to be chosen independently with respect to the resources of the daycare facility and the child’s individual preferences were offered to each child. Examples are given in Table 1. Starting with the ﬁrst predetermined activity ‘drawing’, the child was asked to sit down for ﬁve minutes at a table in a group room and to choose one out of ﬁve coloring pages with different motifs to crayon. All subsequent activities were per- formed continuously for at least three minutes. Additionally, the child could choose two indoor physical activities (e.g., playing with toy cars, toy blocks, hide and seek, playing with dolls). For the performance of the outdoor activities, a well-deﬁned and plain area was chosen. On the outdoors, the child started with a predetermined activity, which was ‘tag’. Afterwards, the child chose two other outdoor activities, depending on the equipment of each of the four local daycare facilities (e.g., swing, tricycle, monkey bars, or scooter). Finally, the child performed the three activities scheduled last, which were walking at normal speed, walking fast, and jogging. During these activities, the researcher accompanied the child to ensure continuous walking and jogging, while the child set the pace. For fast walking, the researcher encouraged the child by telling an imaginary journey of a holiday trip, including catching the tram, train, and plane to get to a holiday location. If the child changed to jogging during the fast walking task, the researcher immediately slowed the child down to fast walking. If the child refused to perform an activity continu- ously due to motivational reasons, the activity was stopped and the child was asked to continue with the next activity. The protocol was completely aborted if the child was exhausted or refused to go on any more.



### Accelerometers

The children were equipped with six accelerometers of three differ- ent brands. One accelerometer was placed at the left and the right wrist (GENEActiv, ActivInsight Ltd, Kimbolton, UK), one acceler- ometer at the left hip (GT3X Actigraph, Pensacola, Florida, USA), two accelerometers at the right hip (GENEActiv, GT3X), and one accelerometer at the right thigh (activPAL, PAL Technologies Ltd, Glasgow, Scotland, UK). Accelerometers at the wrist were ﬁrmly mounted with  $\%$  inch Tyvek security bracelets, which were used instead of the standard bracelets provided by the manufacturer. An elastic belt was used to ﬁx the hip-worn monitors close to the lateral hip bone, and an adhesive pad was used to ﬁx the activPAL to the skin on the front of the right thigh. All accelerometers were operated with the highest possible resolution, which was  $100\,\mathrm{~Hz~}$  in the GENEActiv and the GT3X, and  $20\,\mathrm{~Hz~}$  in the activPAL. Before each measurement we initialized and synchronized the acceler- ometers with the computer running the indirect calorimetry. After completion of the protocol we downloaded accelerometer data from all devices using dedicated software from the manufacturers.

### Physiological Measures

A portable, open-circuit indirect calorimetry system (MetaMax3b, Cortex Biophysics, Leipzig, Germany) was used as the criterion measure. The MetaMax3b has proven reliability in a study with adolescents but was found to slightly overestimate  $\mathrm{VO}_{2}$  during moderate and vigorous exercise (Macfarlane & Wong, 2012). It is highly comparable to other common devices (Brandes, Klein, Ginsel, & Heitmann, 2015). The system was mounted to the children using a pediatric harness. In very small children, addi- tional adhesive tape was used to adjust the harness to the child, so that the system did not interfere with the activities. A facemask was secured over the child´s nose and mouth with an adjustable nylon harness. A bidirectional turbine, inserted to the facemask, mea- sured the volume of inspired and expired air. A sample tube, connected to the turbine, retrieved expired air samples breath-by- breath. Air samples were analyzed for oxygen uptake and carbon dioxide production within the sensor unit of the system. Stable and dry weather conditions enabled sound outdoor measurements. Data were transferred via telemetry to a laptop and available in real-time (MetaSoft 3, Cortex Biophysics, Leipzig, Germany). The laptop operator used the marker function of the software to identify the beginning and the end of each activity. Each morning, prior to the ﬁrst measurement, we calibrated the indirect calorimetry system according to the manufacturer recommended procedure.

Data Reduction and Modeling Oxygen consumption and carbon dioxide production were measured continuously and converted to absolute  $(\mathrm{kJ/min})$  and relative  $\mathrm{(J/min/kg)}$  EE using the equations by Weir (1949). RMR



Note. Data are displayed as  $M\pm S D$  (min, max).

was calculated as the minimum of a rolling one-minute mean during supine rest. Metabolic equivalents (METs) were calcu- lated by dividing the relative oxygen consumption (expressed in  $\mathrm{ml/min/kg)}$  by 3.5.

As done by Montoye et al. (2017), for each individual and device, the following 10 summary statistics were calculated in 30-s non-overlapping windows for each axis  $(\mathrm{X},\mathrm{Y},\mathrm{Z})$  of the raw acceleration data—mean, standard deviation, minimum, maximum, lag one autocorrelation, percentiles  $(10\%,25\%,50\%,75\%,90\%)-$ — resulting in 30 summary statistics for each time window. No ﬁltering of the raw accelerometer data was conducted.

Predictive models were created separately for each acceler- ometer and placement, resulting in 24  $(4\times6)$  models developed and tested. We built four predictive models: (1) a linear regression model, (2) a linear mixed model, (3) a random forest, and (4) an artiﬁcial neural network model. Linear regression models (LM) and linear mixed models (MLM) were created separately for the outcome variables absolute EE (absEE), relative EE (relEE), and METs and included the above mentioned 30 summary statistics as independent variables. The MLM accounts for repeated measure- ments to compute EE prediction.

Random forests (RF) are ensembles of classiﬁcation or regres- sion trees. Each tree is grown on a bootstrap sample of the data to create an ensemble of diverse trees, modelling different aspects of the data. Further randomization is introduced by drawing a random subset of variables at every split. We used regression RFs to predict EE (METs, absEE, or relEE) with raw acceleration data, based on the 30 summary statistics.

Artiﬁcial neural networks (ANNs), inspired by biological neural networks, are widely used to model complex relationships between inputs and outputs. In contrast to the LM and MLM, they do not assume a speciﬁc type of relationship between the outcome and covariates and allow for non-linear dependencies. In this study, ANNs were used to approximate the functional relationship between the response (METs, absEE, or relEE) and the 30 summary statistics as covariates.

All analyses were performed in R (R Core Team, 2017) with the add-on packages ‘lme4’ (Bates, Mächler, Bolker, & Walker, 2015) for MLM, ‘ranger’ (Wright & Ziegler, 2017) for RF, and ‘nnet’ (Venables & Ripley, 2002) for ANN. All model ﬁtting, cross validation, parameter tuning, and evaluation was performed with the ‘mlr’ (Bischl et al., 2016) package. In the RF, we grew 500 trees and tuned the parameter ‘mtry’ using model-based optimization (Bischl et al., 2017). For the ANN we chose a single hidden layer with 15 units and tuned the regularization parameter ‘decay’ to avoid overﬁtting. Code for using the developed models can be found at the following link: https://github.com/bips-hb/EE_prediction.



### Statistical Analysis

Separately for each of the models, accelerometers, and positions, we calculated the root-mean-squared-error (RMSE) to test EE prediction accuracy. We tested all models using a leave-one-out cross-validation.

## Results

We had written informed consent from parents of 62 children. Of these, 41 children completed the protocol. All of the partici- pating children were Caucasians and were able to speak and understand German. Participant characteristics can be found in Table 2.

Reasons for non-participation were withdrawals of the consent on the measurement day (11), illness (5), not being fasted for at least two hours (2), withdrawn after the second activity (1), and missing data due to calorimeter failure (2). Children completed the protocol with a median of eight out of nine possible activities. Mean EE per activity ranged from  $4.2\pm0.9\,\mathrm{kJ/min}$  to  $16.3\pm4.6\,\mathrm{kJ/min}$  (Table 3). Forty  $(98\%)$  children had valid data for the GENEActiv on the left wrist and for the GT3X on the right hip. Thirty-nine  $(95\%)$  children had valid data for the GENEActiv on the right wrist and for the GT3X on the left hip. Thirty-eight  $(93\%)$  children had valid data for the GENEActiv on the right hip and for the activPAL. Predicted METs from the four different models (GENEActiv, left wrist) and METs derived from the indirect calorimeter of one child passing the protocol are exemplarily shown in Figure 1.

Boxplots for RMSE for all accelerometers, locations, and models are shown in Figures 2–4 for absEE, relEE, and METs, respectively.

Mean RMSE ranged from  $2.56{-}2.76\;\mathrm{kJ/min}$  for the RF, from  for the ANN, from  $2.83{-}2.94\,\mathrm{kJ/min}$  $2.72{-}3.08\,\mathrm{kJ/min}$  for the LM, and from  $2.81{-}2.92\;\mathrm{kJ/min}$  for the MLM (Table 4). A comparison of the four models shows that the RF revealed slightly lower RMSE for absEE, relEE and METs than the other three models. GENEActive devices obtained mean RMSE of  $2.56~\mathrm{kJ/min}$  (left and right wrist) and  $2.73\,\mathrm{kJ/min}$  (right hip). Predicting EE using the GT3X on the left and right hip obtained mean RMSE of 2.60 and  $2.74~\mathrm{kJ/min}.$ , respectively. Use of the thigh worn activPAL pro- vided a mean RMSE of  $2.76\,\mathrm{kJ/min}$ . Looking at wear positions, EE prediction accuracy for devices worn at the hip obtained mean RMSE of  $2.60{-}2.74\,\mathrm{\kJ/min}$ . Placement at the wrists obtained slightly lower mean RMSE  $2.56~\mathrm{kJ/min})$ , and placement at the thigh slightly higher mean RMSE  $(2.76~\mathrm{kJ/min})$  (Table 4).



## Discussion

The study’s purpose was to develop and to provide EE prediction models from raw accelerometry data, to compare two linear and two machine learning models, and to compare EE prediction accuracy of different accelerometers placed on the hips, thigh, and wrists. To our knowledge, this is the ﬁrst study that provides a calibration and comparative validation of multiple accelerometers in preschool children aged three to six years against indirect calorimetry as the criterion measurement.



For adults, similar EE prediction accuracy has been reported for linear models vs. an ANN model using different accelerometers (Montoye et al., 2017). Only for wrist-worn devices, Montoye et al. (2017) report a signiﬁcant improvement of the ANN over linear models. In children, there is preliminary evidence that the applica- tion of machine learning improves EE prediction accuracy com- pared to linear regression algorithms (Chowdhury et al., 2018; Mackintosh et al., 2016). In our study, machine learning models and (mixed) linear models were equally accurate in EE prediction irrespective of the device and wear position. This could be due to the fact, that our models all included the same summary statistics as independent variables whereas in the simpliﬁed approach by Chowdhury et al. (2018) only three features, including mean  $(x)$ , mean (y), mean (z), were extracted from each accelerometer window. Thus our data shows preliminary evidence that applica- tion of an RF or ANN model leads to only minor improvements in EE prediction compared to (mixed) linear models if the models are built upon the same summary statistics as independent vari- ables. We can therefore recommend using linear as well as non-linear models for EE prediction on raw accelerometry data in studies that aim at assessing young children’s EE under free-living conditions. Furthermore, comparison of mean RMSE did not reveal meaningful differences between the three different devices. All of them can therefore be recommended for use in young children.



Accelerometers are commonly worn on a waist belt, aligned with the right anterior axillary line for the entire day up to seven days to estimate habitual PA (Hills et al., 2014). To date, most research has used the hip-worn GT3X, which has been calibrated and validated in a wide range of populations (Borghese et al., 2017; Evenson et al., 2015; Johansson, Larisch, Marcus, & Hagstromer, 2016). Compliance among young children has been shown to be better when using wrist-worn accelerometers compared to hip- worn devices (Fairclough et al., 2016). Therefore, our main aim was to test whether EE prediction of wrist-, hip- and thigh-worn devices is equally accurate for age-appropriate activities in pre- schoolers. Our results show good accuracy for all wear positions. This is in line with the ﬁndings of another study with children in which accelerometers mounted on various anatomical positions demonstrated equivalency in the accuracy to predict EE in a semi- structured setting (Mackintosh et al., 2016).

In our study, we further observed only small differences regarding the predictive accuracy between the dominant and non-dominant wrist. This also is in line with the ﬁndings of another study that examined the classiﬁcation accuracy of the GENEActiv with a cut-point based approach for the assessment of PA intensi- ties in preschoolers (Roscoe, James, & Duncan, 2017) and may help to inform choice of wrist placement when using acceler- ometers in ﬁeld-based research. We assume good accuracy in predicting EE should the accelerometer either be placed on the left or right wrist.

There are several strengths of our study which are in line with best practice recommendations for calibration studies (Welk, 2005). We applied a semi-structured nature of the activity protocol, numerous accelerometer positions, and a high-quality criterion measure for EE. Our sample size of 41 children is in line with recent calibration and validation studies and the included activities enabled their differentiation into light- to vigorous intensity activi- ties quite well, with the lowest activity-speciﬁc metabolic equiva- lents (AME) being observed in drawing [mean  $(95\%$  CI): 1.89 (1.7–2.1)], a moderate AME in playing with cars [3.1 (2.5–3.6)] and walking [3.6 (3.3–4.0)], and the most vigorous AME in jogging [7.7 (6.7–8.7)] (Brandes, Steenbock, & Wirsik, 2018). The generalizability of our results is indeed limited by the homog- enous sample of apparently healthy Caucasian children. Therefore, our ﬁndings cannot be generalized to other ethnicities or to children with a chronic condition or disease. A further limitation of the study is that not all children completed the ﬁve predetermined activities that stretch across all activity intensities (drawing, catching, walk- ing at regular speed, walking fast, and jogging). This was due to the young age and motivational reasons in our target group.



## Conclusion

These ﬁndings provide preliminary evidence that recent acceler- ometers mounted on ﬁve different anatomical positions demon- strate equivalency in the accuracy to predict EE from raw accelerometer data. Our models are publicly available and can be used for studies assessing EE in preschoolers under free-living conditions. The RF, ANN, and (mixed) linear models were equally accurate in predicting EE irrespective of the device and wear position. We therefore recommend utilizing linear as well as non-linear models for the estimation of EE in preschoolers. We recommend a further investigation of RF models for EE prediction from raw accelerometer data in young children as well as in other populations, which seems to be a viable alternative to linear and ANN models.

Note. LM $=$ linear model; MLM $=$ mixed linear model;  $\mathrm{RF}=1$ random forest; ANN $=$ artiﬁcial neural network; RMSE $=$ root-mean-square error.

### Acknowledgments

The study was supported by an internal innovations grant of the Leibniz Institute for Prevention Research and Epidemiology (BIPS). We thank Johanna Sophie Lubasch und Marlena Böning for their assistance during data collection and all daycare facilities, daycare staff, parents and children for their support and participation in the study. The authors declare that they have no conﬂicts of interest.

## References

Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. doi:10.18637/jss.v067.i01 Bischl, B., Lang, M., Kotthoff, L., Schiffner, J., Richter, J., Studerus, E., : : : Jones, Z.M. (2016). mlr: Machine Learning in R. The Journal of Machine Learning Research, 17(170), 1–5.



Bischl, B., Richter, J., Bossek, J., Horn, D., Thomas, J., & Lang, M. (2017). mlrMBO: A Modular Framework for Model-Based Optimi- zation of expensive black-box functions. Retrieved from http://arxiv. org/abs/1703.03373 Borges, J.H., Langer, R.D., Cirolini, V.X., Pascoa, M.A., Guerra-Junior, G., & Goncalves, E.M. (2016). Minimum time to achieve the steady state and optimum abbreviated period to estimate the resting energy expenditure by indirect calorimetry in healthy young adults. Nutrition in Clinical Practice, 31(3), 349–354. PubMed ID: 26888859 Borghese, M.M., Tremblay, M.S., LeBlanc, A.G., Leduc, G., Boyer, C., & Chaput, J.P. (2017). Comparison of ActiGraph GT3X+ and actical accelerometer data in 9–11-year-old Canadian children. Journal of Sports Sciences, 35(6), 517–524. PubMed ID: 27103499 doi:10. 1080/02640414.2016.1175653 Brandes, M., Klein, H., Ginsel, S., & Heitmann, A. (2015). Comparability of three mobile respiratory gas analyzers. Sportwissenschaft, 45(4), 168–172. doi:10.1007/s12662-015-0368- $\mathrm{X}$  Brandes, M., Steenbock, B., & Wirsik, N. (2018). Energy cost of common physical activities in preschoolers. Journal of Physical Activity & Health, 15(4), 233–238. PubMed ID: 29202634 doi:10.1123/jpah. 2017-0348 Butte, N.F., Wong, W.W., Lee, J.S., Adolph, A.L., Puyau, M.R., & Zakeri, I.F. (2014). Prediction of energy expenditure and physical activity in preschoolers. Medicine & Science in Sports & Exercise, 46(6), 1216– 1226. PubMed ID: 24195866 doi:10.1249/mss.0000000000000209 Caspersen, C.J., Powell, K.E., & Christenson, G.M. (1985). Physical activity, exercise, and physical ﬁtness: Deﬁnitions and distinctions for health-related research. Public Health Rep, 100(2), 126–131. PubMed ID: 3920711 Chowdhury, A.K., Tjondronegoro, D., Zhang, J., Hagenbuchner, M., Cliff, D., & Trost, S.G. (2018). Deep learning for energy expen- diture prediction in pre-school children. Paper presented at the IEEE Conference on Biomedical and Health Informatics, Las Vegas, NA.

      

     

    

   

  

 



Davies, G., Reilly, J., McGowan, A., Dall, P., Granat, M., & Paton, J. (2012). Validity, practical utility, and reliability of the activPAL in preschool children. Medicine & Science in Sports & Exercise, 44(4), 761–768. PubMed ID: 21983077 De Decker, E., De Craemer, M., Santos-Lozano, A., Van Cauwenberghe, E., De Bourdeaudhuij, I., & Cardon, G. (2013). Validity of the ActivPAL and the ActiGraph monitors in preschoolers. Medicine & Science in Sports & Exercise, 45(10), 2002–2011. PubMed ID: 23524516 doi:10.1249/MSS.0b013e318292c575 Evenson, K.R., Wen, F., Herring, A.H., Di, C., LaMonte, M.J., Tinker, L.F.,: : :Buchner, D.M. (2015). Calibrating physical activity intensity for hip-worn accelerometry in women age 60 to 91 years: The Women’s Health Initiative OPACH Calibration Study. Preven- tive Medicine Reports, 2, 750–756. PubMed ID: 26527313 doi:10. 1016/j.pmedr.2015.08.021 Fairclough, S.J., Noonan, R., Rowlands, A.V., Van Hees, V., Knowles, Z., & Boddy, L.M. (2016). Wear compliance and activity in children wearing wrist- and hip-mounted accelerometers. Medicine & Science in Sports & Exercise, 48(2), 245–253. PubMed ID: 26375253 doi:10. 1249/mss.0000000000000771 Hagenbuchner, M., Cliff, D.P., Trost, S.G., Van Tuc, N., & Peoples, G.E. (2015). Prediction of activity type in preschool children using machine learning techniques. Journal of Science and Medicine in Sport, 18(4), 426–431. PubMed ID: 25088983 Hills, A.P., Mokhtar, N., & Byrne, N.M. (2014). Assessment of physical activity and energy expenditure: An overview of objective measures. Frontiers in Nutrition, 1, 5. PubMed ID: 25988109 doi:10.3389/fnut. 2014.00005 Janssen, X., Cliff, D.P., Reilly, J.J., Hinkley, T., Jones, R.A., Batterham, M., : : : Okely, A.D. (2013). Predictive validity and classiﬁcation accuracy of ActiGraph energy expenditure equations and cut-points in young children. PLoS One, 8(11), e79124. PubMed ID: 24244433 doi:10.1371/journal.pone.0079124 Jimmy, G., Seiler, R., & Maeder, U. (2013). Development and validation of energy expenditure prediction models based on GT3X accelerom- eter data in 5- to 9-year-old children. Journal of Physical Activity & Health, 10(7), 1057–1067. PubMed ID: 23136371 Johansson, E., Larisch, L.M., Marcus, C., & Hagstromer, M. (2016). Calibration and validation of a wrist- and hip-worn actigraph accel- erometer in 4-year-old children. PLoS One, 11(9), e0162436. PubMed ID: 27617962 doi:10.1371/journal.pone.0162436 Knaeps, S., Bourgois, J.G., Charlier, R., Mertens, E., Lefevre, J., & Wijndaele, K. (2018). Ten-year change in sedentary behaviour, moderate-to-vigorous physical activity, cardiorespiratory ﬁtness and cardiometabolic risk: Independent associations and mediation analysis. British Journal of Sports Medicine, 52(16), 1063–1068. PubMed ID: 27491779 doi:10.1136/bjsports-2016-096083 Lamonte, M.J., & Ainsworth, B.E. (2001). Quantifying energy expendi- ture and physical activity in the context of dose response. Medicine & Science in Sports & Exercise, 33(Suppl. 6), S370–S378. discussion S419–S320. PubMed ID: 11427762 Macfarlane, D.J., & Wong, P. (2012). Validity, reliability and stability of the portable cortex metamax 3B gas analysis system. European Journal of Applied Physiology, 112(7), 2539–2547. PubMed ID: 22075643 doi:10.1007/s00421-011-2230-7 Mackintosh, K.A., Montoye, A.H., Pfeiffer, K.A., & McNarry, M.A. (2016). Investigating optimal accelerometer placement for energy expenditure prediction in children using a machine learning approach. Physiological Measurement, 37(10), 1728–1740. PubMed ID: 27653339 doi:10.1088/0967-3334/37/10/1728 Migueles, J.H., Cadenas-Sanchez, C., Ekelund, U., Delisle Nystrom, C., Mora-Gonzalez, J., Lof, M., : : : Ortega, F.B. (2017). Accelerometer data collection and processing criteria to assess physical activity and other outcomes: A systematic review and practical considerations. Sports Medicine, 47(9), 1821–1845. PubMed ID: 28303543 doi:10. 1007/s40279-017-0716-0 Montoye, A.H., Begum, M., Henning, Z., & Pfeiffer, K.A. (2017). Comparison of linear and non-linear models for predicting energy expenditure from raw accelerometer data. Physiological Measure- ment, 38(2), 343–357. PubMed ID: 28107205 doi:10.1088/1361- 6579/38/2/343 Montoye, A.H., Mudd, L.M., Biswas, S., & Pfeiffer, K.A. (2015). Energy expenditure prediction using raw accelerometer data in simulated free living. Medicine & Science in Sports & Exercise, 47(8), 1735–1746. PubMed ID: 25494392 doi:10.1249/mss.0000000000000597 Nilsson, A., Brage, S., Riddoch, C., Anderssen, S.A., Sardinha, L.B., Wedderkopp, N., : : : Ekelund, U. (2008). Comparison of equations for predicting energy expenditure from accelerometer counts in children. Scandinavian Journal of Medicine & Science in Sports, 18(5), 643–650. PubMed ID: 18208433 doi:10.1111/j.1600-0838. 2007.00694.x R Core Team. (2017). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/ Reddon, H., Meyre, D., & Cairney, J. (2017). Physical activity and global self-worth in a longitudinal study of children. Medicine & Science in Sports & Exercise, 49(8), 1606–1613. PubMed ID: 28319588 doi:10. 1249/mss.0000000000001275 Reilly, J.J., Kelly, L.A., Montgomery, C., Jackson, D.M., Slater, C., Grant, S., & Paton, J.Y. (2006). Validation of Actigraph accelerometer estimates of total energy expenditure in young children. International Journal of Pediatric Obesity, 1(3), 161–167. PubMed ID: 17899634 Roscoe, C.M.P., James, R.S., & Duncan, M.J. (2017). Calibration of GENEActiv accelerometer wrist cut-points for the assessment of physical activity intensity of preschool aged children. European Journal of Pediatrics, 176(8), 1093–1098. PubMed ID: 28674825 doi:10.1007/s00431-017-2948-2 Shoup, J.A., Gattshall, M., Dandamudi, P., & Estabrooks, P. (2008). Physical activity, quality of life, and weight status in overweight children. Quality of Life Research, 17(3), 407–412. PubMed ID: 18293100 doi:10.1007/s11136-008-9312-y Skotte, J., Korshoj, M., Kristiansen, J., Hanisch, C., & Holtermann, A. (2014). Detection of physical activity types using triaxial acceler- ometers. Journal of Physical Activity & Health, 11(1), 76–84. PubMed ID: 23249722 doi:10.1123/jpah.2011-0347 Skrede, T., Stavnsbo, M., Aadland, E., Aadland, K.N., Anderssen, S.A., Resaland, G.K., & Ekelund, U. (2017). Moderate-to-vigorous physical activity, but not sedentary time, predicts changes in cardiometabolic risk factors in 10-y-old children: The active smarter kids study. The American Journal of Clinical Nutrition, 105(6), 1391–1398. PubMed ID: 28381476 doi:10.3945/ajcn.116. 150540 Venables, W.N., & Ripley, B.D. (2002). Modern Applied Statistics with S (4th ed.). New York, NY: Springer.

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 



Wafa, S.W., Shahril, M.R., Ahmad, A.B., Zainuddin, L.R., Ismail, K.F., Aung, M.M., & Mohd Yusoff, N.A. (2016). Association between physical activity and health-related quality of life in children: A cross- sectional study. Health and Quality of Life Outcomes, 14, 71. PubMed ID: 27146199 doi:10.1186/s12955-016-0474-y Weir, J.B. (1949). New methods for calculating metabolic rate with special reference to protein metabolism. The Journal of Physiology, 109(1–2), 1–9. PubMed ID: 15394301 Welk, G.J. (2005). Principles of design and analyses for the calibration of accelerometry-based activity monitors. Medicine & Science in Sports & Exercise, 37(Suppl. 11), S501–S511. PubMed ID: 16294113 Wright, M.N., & Ziegler, A. (2017). Ranger: A fast implementation of random forests for high dimensional data in  $C{++}$  and R. Journal of Statistical Software, 77(1), 1–17. doi:10.18637/jss.v077.i01

  

 


# On 3-Extra Connectivity and 3-Extra Edge

# Connectivity of Folded Hypercubes

Nai-Wen Chang, Cheng-Yen Tsai, and Sun-Yuan Hsieh, Senior Member, IEEE

Abstract—Given a graphand a non-negative integer $g,$ , the $g_{\parallel}$ -extra connectivity (resp. $g_{\parallel}$ -extra edge connectivity) ofis the minimum cardinality of a set of vertices (resp. edges) in, if it exists, whose deletion disconnectsand leaves each remaining component with more thanvertices. This study shows that the 3-extra connectivity (resp. 3-extra edge connectivity) of an-dimensional folded hypercube isfor(resp.for). This study also provides an upper bound for the $g_{\parallel}$ -extra connectivity on folded hypercubes for.

     

    

   

  

 



Index Terms—Interconnected networks, connectivity, extra connectivity, extra edge connectivity, fault-tolerance, reliability

1INTRODUCTION BECAUSE of the rapid development of VLSI technology, a multipro- cessor system may contain hundreds or even thousands of nodes, some of which may be faulty when the system is implemented. As the number of processors in a system increases, the possibility that its processors may become faulty also increases. Because designing such systems without defects is nearly impossible, and fault tolerance are two of the most critical concerns of multiprocessor systems [19].



Based on the deﬁnition proposed by Esfahanian [1], a multi- processor system is fault tolerant if it can remain in the presence of failures. Two basic functionality criteria have received considerable attention. The ﬁrst criterion for a system to be re- garded as functional is whether the network logically contains a certain topological structure. This is the problem that occurs when embedding one architecture into another [11], [18]. This approach involves using system-wide redundancy and reconﬁguration. The second functionality criterion considers a multiprocessor system functional if a fault-free communication path1 exists between any two fault-free nodes; that is, the topological structure of the multiprocessor system remains connected in the presence of cer- tain failures. Thus, and edge connectivity are two major measurements of this criterion [18]. The connectivity of a graph $G,$  denoted by $\kappa(G),$  is the minimal number of vertices whose removal fromproduces a disconnected graph or only one vertex; the edge connectivity of a graph $G,$  denoted by $\lambda(G),$  is the minimal number of edges whose removal fromproduces a disconnected graph.

    

   

  

 



However, these two parameters tacitly assume that all vertices that are adjacent to, or all edges that are incident to, the same vertex can potentially fail simultaneously. This is practically impossible in some network applications. To address this deﬁciency, two speciﬁc terms forbidden faulty set and forbidden faulty edge set are introduced. The vertices in a forbidden faulty set or the edges in a forbidden faulty edge set cannot fail simultaneously. By restricting the for- bidden faulty set (resp. forbidden faulty edge set) to the sets of neighbors (resp. neighboring edges) of any spanning subgraph with no more than $g$ -vertices in the faulty networks, Fàbrega and Fiol [5], [6] introduced the following measures. Given a graphand a non-negative integer $g,$  the $g$ -extra connectivity (resp. $g_{-}$ -extra edge connectivity) of $G,$  written as(resp. $\lambda_{g}(G)),$  is the minimal cardinality of a set of vertices (resp. edges) in $G,$  if it exists, whose deletion disconnectsand leaves each remaining component with more thanvertices.

    

   

  

 



Note thatandfor any graphifis not a complete graph. Therefore, the-extra connectivity (resp.



-extra edge connectivity) can be regarded as a general form of the classical connectivity (resp. classical edge connectivity) that pro- vides measures that are more accurate for reliability and fault- tolerance for large-scale parallel processing systems. Regarding the computational complexity of the problem, based on thorough research, no polynomial-time algorithm has been presented to computefor a general graph; nor has there been any tight upper bound for[2]. However,can be computed by solving numer- ous network ﬂow problems [3]. The $g$ -extra connectivity and $g_{\l}$ -extra edge connectivity of particular classes of graphs have recently received considerable attention [2], [3], [5], [6], [10], [16], [17], [20]-[23], [25], [26].

 



The hypercube is one of the most recognized interconnected net-

### works. An-dimensional hypercubeis an-regular graph with

vertices andedges. Each vertexincan be labeled by an -bit binary string. Two vertices are joined by an edge if and only if their binary strings differ in a one-bit position only. The recursive deﬁnition ofis as follows:is a complete graph with two vertices labeled by 0 and 1. Letbe a “don’t care” symbol. For $n\geq2,$ is constructed from two copies of  $(n-1)\cdot$ )- dimensional hypercubes (subcubes), namelyand, where is a subgraph ofinduced by vertex subsetand is a subgraph ofinduced by vertex subset. Edges exist between corresponding vertices in the two subcubes such that vertexinis adjacent to vertexin. The folded hypercube is an attractive variation of the hypercube pro- posed by El-Amawy and Latiﬁ [1]. An-dimensional folded hypercube can be constructed from an $n$ -dimensional hypercube by adding edges, called “complementaryedges”, between opposite pairs of verticesand $\overline{{u}}=\overline{{u}}_{1}\overline{{u}}_{2}\ldots\overline{{u}}_{n},$ where $\overline{{u}}_{i}=1-u_{i}.F Q_{n}$ is an  $(n+1)$ )-regular  $(n+1)\cdot$ )-connected graph withvertices and edges. The folded hypercube not only preserves the advantages of the hypercube, such as its symmetric properties, but also possesses properties that are superior to those of the hypercube. The topological properties of folded hypercubes have been exten- sively investigated [1], [8], [12], [20], [25], [26]. Folded hypercubes have been used as underlying topologies of several parallel systems, such as ATM switches [13], [14], PM2I networks [9], and 3D-FolH- NOC networks [4], for high-speed cell-switching and reducing the diameter and trafﬁc congestion of the hypercube with little hardware overhead.

             

            

           

          

         

        

       

      

     

    

   

  

 



The probability that a vertex or an edge fails is called “failing probability”. For simplicity, assume that each vertex (resp. edge) possesses the same failing probability, denoted by $p\left(\operatorname{resp.}q\right)$ . Further, let(resp. $\xi_{1}^{\prime},\xi_{2}^{\prime},\xi_{3}^{\prime})$  denote the events that all of the neighbors (resp. neighboring edges) of any vertex, any edge, any path of 3 vertices infail simultaneously, respectively. Their probabilities aregivenby $p r o b(\xi_{1})\leq2^{n}\cdot p^{n+1},$ , ,, ,for. It can be seen that these probabilities become relatively small for sufﬁ ciently large. For example, by adoptingand $n=5,$  theprobabilityofisolatingacomponentwithatmost3 vertices is calculated as

        

       

      

     

    

   

  

 



, which is rela tively small. Therefore, the probability of isolating a component with at most three vertices is extremely small (i.e., the probabilities of all of the neighbors (neighboring edges) of any vertex, any edge, or any path of 3 vertices failing simultaneously inis extremely small).



Therefore, consideringandis reasonable and meaningful.Previousresultshaveshownthat for[1],for [1],for[20],for [25],for[26], andfor [26]. This paper shows thatforand for. This paper also provides an upper bound offor.

      

     

    

   

  

 



The remainder of this paper is organized as follows. Section 2 introduces necessary deﬁnitions and notations. Section 3 provides basic properties related to bothand. Section 4 explores the 3-extra connectivity and 3-extra edge connectivity of. Finally, Section 5 presents the conclusion of the paper.

  

 



2PRELIMINARIES A graphis a pair of the vertex setand the edge set $E,$  whereisaﬁnitesetandisasubsetof

 



. Useandto denote the vertex set and edge set of, respectively. This study considers undirected, simple, and connected graphs. Two verticesandare

 



## adjacent if $(u,v)\in E(G),$  andandare end-vertices of  $(u,v)$ ). A

subgraphof, denoted by $H\subseteq G,$  is a graph whereand. The components ofare its maximal connected subgraphs.

 



For any vertex $u\operatorname{in}G,$  $G,$  the neighborhood(neighbors) of $u\operatorname{in}G,$ , denoted by, is the set of vertices adjacent toin. The edge neighborhood (neighboring edges) ofin $G,$  denoted by $N E_{G}(u),$  is the set of edges incident toin. The degree of a vertexin $G,$  denoted by $d e g_{G}(u),$  is thecardinality of. Agraphis -regular ifforevery vertexin. Letbe any vertex subset of; that is,. The

    

   

  

 



### neighborhood and edge-neighborhood ofinare deﬁned as

### , respectively. The closed neighborhood ofinis

deﬁned as. For a subgraphin, the notation (resp.) can be simpliﬁed as(resp.



). The notationcan be simpliﬁed as.

A pathforin a graphis a sequence of distinct vertices such that any two consecutive vertices are adjacent. A cycleforis a sequence of vertices in which any two consecutive vertices are adjacent, where are all distinct. The girth ofis the length of the shortest cycle in. A complete graphis a graph consisting ofpairwise adjacent vertices.

  

 



A complete bipartite graphis a graph consisting of two vertex setsand $V_{\mathrm{{2}}},$  called partite sets, with ordersand $s,$  respectively. Two vertices are adjacent if and only if they are in different partite sets (i.e., for every pairand $y\in V_{2},$  an edge  $(x,y)$ ) is in). Useto denote the distance of two verticesand

   

  

 



### in.Thediameterofisdeﬁnedas

. An isomorphism from a simple graph to a simple graphis a one-to-one and an onto function suchthatifandonlyif

 



## . Note that  $^{\prime\prime}G$ is isomorphic to written as $H,^{\prime\prime}$ 

, if there is an isomorphism fromto. Letbe a non- negative integer and  $\mathcal{P}_{g}$ be the property of having more than vertices. Given a vertex subset (resp. edge subset)in a graph

 



, use the notationto represent the subgraph of obtained after deleting all vertices (resp. edges) infrom. For convenience, refer to the vertices (edges) inas faulty vertices (resp.

 



faulty edges). A faulty vertex subsetis an  $\mathcal{R}_{g}$ -cut ifis disconnected and every remaining component ofhas property . A faulty edge subset $\mathcal{P}_{g}$ is a (- $\mathcal{R}_{g}^{\prime}.$  cut for short) ifis disconnected and every remaining component of

  

 



## has property  $\mathcal{P}_{g}$ . The-extra connectivity of $G,$  denoted by

, equals $\mathcal{R}_{g}$ . Similarly, the $g$ -extra edge

## connectivity of $G,$  denoted by $\lambda_{g}(G),$  equals

R.

 $\mathrm{An}\,(n+1)\!.$ )-dimensional folded hypercubecan be regarded as a graph consisting of two-dimensional hypercubesand $Q_{n}^{1},$  denoted by $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  whereandare two-dimen- sional hypercubes with the preﬁx bits 0 and 1 in each vertex, respectively.Thesymbolrepresentstheperfectlymatched operation that connectsandby using two speciﬁc perfect matchingsand

    

   

  

 



. For convenience, also use the notationto represent. In addition, use  $0u\,(\mathrm{resp.}\,1u)$ ) to represent a vertex ofbelonging to(resp.

 



). Any vertexhas two neighbors and any vertexhas two neighbors. The girth ofequals 4 for[26].

 



For any two verticesand $y,$  useto denote the Hamming distance betweenand $y;$  that is, the number of different positions between the binary strings ofand $y$ . A summary of notations and symbols is listed in Table 1.

  

 



3BASIC PROPERTIES Lemmas 1-3 can be easily veriﬁed becauseis-regular and triangle-free for every.

 



### Lemma1. If()admitsasasubgraph,then

.

## Lemma2. If $(n\geq3)$ )admitsasasubgraph,then

### if the end-vertices ofare adjacent, and

otherwise.

## Lemma3. If $(n\geq3)$ )admitsasasubgraph,then

.

is ()-regular and triangle-free forbecause the girth equals 4; therefore, the following lemma is derived.



## Lemma 4. Letbe a subgraph of $F Q_{n+1},$  where. Then,

.

### Lemma 5. [22] Letbe any subset of the vertices ofwith

, where. Then,.

## Lemma 6. [22] Letbe a subgraph of $Q_{n},$  where

### and.Then,thefollowingtwostatementshold:

(1). (2)is connected with property  $\mathcal{P}_{g}$ .



## Lemma 7.  $\it{122}J\it{L e t}V^{\prime}$ be a vertex subset inwith $|V^{\prime}|=g+1,$  where

### and. Then,is connected with

property  $\mathcal{P}_{g}$ .

## Lemma 8. Let $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  where $n\geq5,$  and letbe a

### connectedsubgraphof.If $d i a m(A)\leq n-1,$ then

.

Proof. Letandbe any two vertices in. Ifandhave common neighbors in $Q_{n}^{1},$  thenor $1\overline{{u}}=1v\;(\mathrm{i.e.,}\,0u=0\overline{{v}}),$ ), whichleadstoacontradictionbecause

 



. In other words, any two distinct vertices inhave no common neighbors in.



Furthermore, because each vertex inhas two neighbors in $Q_{n}^{1},$  . ◽



## Lemma 9. Let $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  where $n\geq5,$  and letbe a

### subgraphof $Q_{n}^{0},$ where.Then,thefollowingtwo

statementshold:(1).

(2)is connected with property  $\mathcal{P}_{g}$ .

Proof. By using Lemmas 6 and 8, Thus, Condition 1 holds.



By using Lemmas 6 and 7 (by lettingin Lemma 7), bothandareconnected.



Because forand,is connected to , which leads tobeing con- nected. In addition, because forand, has property  $\mathcal{P}_{g}$ . ◽ 4MAIN RESULTS

     

    

   

  

 



## 4.13-Extra Connectivity

## Lemma10. Let,andlet $F\subset V(F Q_{n+1}),$ 

## , and $F_{1}=F\cap V(Q_{n}^{1}),$  where. If

## anddoes not contain isolated $K_{1},$  isolated $K_{2},$  or

## isolated $P_{3},$  then every vertex inis connected to  $a$  vertex in

under.

Proof. Letbe any vertex in(see Fig. 1). Ifor , then the result holds. Hence, assume thatand . Becausecontains no isolated $K_{1},$ has a neighborin. Ifor $1\overline{{v}}\notin F_{1},$  then the result holds.

  

 



Hence, assume thatand. Because contains no isolated $K_{2},$  eitherorhas a neighborin . Ifor $1\overline{{w}}\notin{\cal F}_{1},$  then the result holds. Hence, assume thatand. Becausecontains no isolated $P_{3},\{0u,0v,0w\}$ has a neighborin. If or $1{\overline{{x}}}\notin F_{1},$  then the result holds. Therefore, assume that and. For convenience, letand

     

    

   

  

 



.

Letbe the connected subgraph induced by $V^{\prime}\operatorname{in}Q_{n}^{0}$ . Because and $d i a m(A^{\prime})\leq3,$  all vertices inare distinct (i.e., ). Let. Letbe any vertex inand letbe the connected subgraph induced by (in $Q_{n}^{0})$ . Becauseand $d i a m(A^{\prime\prime})\leq4,\ 1y\notin{\cal F}^{\prime},$ , implying that only vertices incan be the faulty vertices in. Because(by using Lemma 5) and contains at mostfaulty verti- ces, an edgeexists such that (1)(i.e., ), and (2)(i.e., $1y\notin F_{1}-F^{\prime}),$  implying that because. Therefore,can be connected to a vertex in. ◽ Theorem 1.for.

           

          

         

        

       

      

     

    

   

  

 



Proof.forisdemonstratedby constructing an  $\mathcal{R}_{3}$ -cut of sizeinas follows. Let , and letbe a connected subgraph of.

 



Using Lemma 9 then yields (1)

, and (2)is connected with property  $\mathcal{P}_{3}$ . Therefore,is the desired  $\mathcal{R}_{3}$ -cut, implying thatfor.

 



Next, provethatfor. In other words, the size of every  $\mathcal{R}_{3}$ -cut ofis at least. For the sake of contradiction, supposewithis an -cut; that is, $\mathcal{R}_{3}$ does not contain isolated $K_{1},$  isolated , or isolated. Showing thatis connected yields a contradiction.

    

   

  

 



Letand. Without loss of generality, assume that. Thus,



, implyingfor.

According to the result in [7],then satisﬁes one of the following four scenarios: (1)is connected, (2) contains two components, one of which one is an isolated vertex, (3)contains two components, one of which is an isolated edge. (4)contains three components, two of which are isolated vertices.

   

  

 



Case 1.is connected. Because every vertex inis connected to a vertex in(by using Lemma 10), is connected.

 



Case 2.contains two components, one of which is an isolated vertex; that is,. It remains to show thatcan be connected toby using some vertices in .Becausecontainsnoisolated $K_{1},$  or. If $1\bar{t}\notin F_{1},$  then the result holds. Hence, assume that. Consider the following two subcases.

    

   

  

 



Case 2.1.and. Becausecontains no isolated $P_{3},$  eitherorhas a neighborin. If or $1{\overline{{u}}}\not\in F_{1},$  then the result holds. Hence, assume that and. Without loss of generality, assume that is adjacent to. For the neighbors ofin $Q_{n}^{0},$  the following two scenarios are considered further: Case 2.1.1 $N_{Q_{n}^{0}}(0\bar{t})\not\subset F_{0}$ . Letbe a vertex in (seeFig.2a).Ifor $1{\overline{{v}}}\notin F_{1},$ thentheresult holds. Hence, assume thatand. Because ,, imply- ing

         

        

       

      

     

    

   

  

 



. Clearly,. Because andcontainsatmost faulty vertices, an edgeexists such thatand.

  

 



## Case 2.1.2. Because $d_{H}(0t,0\bar{t})=d_{H}(1t,1\bar{t})=n\geq5,$ 

and

. Therefore, because andcontainsatmost faulty neighbors, eitherorhas a fault-free neighborin (seeFig.2b).Let

   

  

 



.Clearly,.

Because(by using Lemma 5) andcontains at most faulty vertices, an edge exists such thatand.

  

 



Case 2.2. (and) or (and $0\Bar{t}\not\in F_{0})$ . Without loss of generality, assume thatand. If $1{\bar{t}}\not\in F_{1},$  then the result holds. Hence, assume that. Because contains no isolated $K_{2},\,0t$ has a neighborin (see Fig. 3). Ifor $1{\overline{{u}}}\notin F_{1},$  then the result holds.

   

  

 



Therefore,assumethatand.Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $1{\overline{{v}}}\notin F_{1},$  then the result holds. Hence, assume thatand. Because ,and , implying and .Because(byusing Lemma 5) andcontains at most faulty neighbors in $Q_{n}^{0},$ has a neighborin. Ifor $1\overline{{w}}\notin F_{1},$  then the result holds. Therefore, assume thatand.

          

         

        

       

      

     

    

   

  

 



Forconvenience, letand

. The remaining proof is similar to the method described in Paragraph 2 in the proof of Lemma 10.



Case 3.contains two components, one of which is an isolated edge; that is,  $(1u,1v)$ ). Furthermore,  $(1u,1v)$ ) can be connected toby using some vertices in

 



. Becauseand

,or. First consider the case that and. Becausecontains no isolated  $K_{2},\ \left|\{0u,0\overline{{{u}}},0v,0\overline{{{v}}}\}\cap F_{0}\right|\neq4$ . Hence,  $(1u,1v)$ ) can be con- nected toby using vertices inand the result holds. Next, consider the case thatand (orand). Ifor $0{\overline{{v}}}\notin F_{0},$  then the result holds. Hence, assume thatand. Because contains no isolated,or.

      

     

    

   

  

 



Notethatbecauseand (because),



. Furthermore, because and $|F_{0}|\geq|F_{1}|=2n-1,|F_{0}|=|F|-|F_{1}|=2n-1.$ . This leads to the following scenarios.

 



can be connected to a vertex inby using the bold edges.

inand, at most one isolated vertex exists in . Conversely, because at leastedges must be removed to obtain an isolated vertex subsetin, where , andfor,satisﬁes one adjacent to. Thus,is also adjacent to. Because is adjacent toand $d_{H}(1\overline{{u}},1u)\geq5,$ , which leads tobecause. Therefore,  $(1u,1v)$ ) can be connected to a vertex inby using and.

       

      

     

    

   

  

 



### Case 3.2. (and) or (and). With-

out loss of generality, assume thatand. Recall that. Becausecontains no isolated $P_{\mathrm{{3}}},$ has a neighborin(see Fig. 4b). The remainder of the proof is similar to that used in Case 3.1.

  

 



Case 4.contains three components, two of which are isolated vertices; that is,and. Based on the method similar to that used in Case 3, we can show thatand can be connected toby using some vertices in.

   

  

 



Combining the above cases completes the proof. ◽ Based on Lemma 9 and the proof of Theorem 1, the following corollary extends the results to suggest an upper bound of.

 



## Corollary 1., whereand

.

## 4.23-Edge Extra Connectivity

## Lemma 11. Letbe a connected subgraph ofwith $|V(A)|=4,$ 

### where. Then, each neighbor ofinis adjacent to at most two

vertices in.

Proof. Note thatcan be isomorphic to $P_{4,}$ or. Letbe a neighbor ofin. Suppose, conversely, thatis adjacent to at least three vertices in. Hence, either (1) a triangle is generated, or (2) two vertices having at least three common neighbors exist in , which leads to a contradiction. ◽ Lemma 12. Letwith $|F|\leq2n-2,$  where. Then,

    

   

  

 



### satisﬁes one of the following three scenarios: (1)

### is connected. (2)has two components, one of which is an

### isolated vertex. (3)has two components, one of which is an

isolated edge.

Proof. Ifis connected, then Condition 1 holds. Otherwise, assume thatis disconnected. Because



[25], where,contains at least one isolated vertex or one isolated edge. In addition, because at of the following two scenarios: (i) There is exactly one isolated vertex (hereafter, denoted by ) in. (ii) There is exactly one isolated edge, hereafter denoted by  $(u,v),$ ), in.

  

 



If (i) holds, thenis connected as follows.

Becauseand ,is connected. Hence, Condition 2 holds. If (ii) holds, then is connected as follows. Because forand ,is connected. Hence, Condition 3 holds. ◽

     

    

   

  

 



## Lemma13. Let.Let

### ,,,and

### , where. Suppose thatand

## does not contain any isolated $K_{1},$  isolated $K_{2},$  or

## isolated. Then, every vertex inis connected to  $a$  vertex in

under.

Proof. Letbe any vertex in(see Fig. 5). Ifor ,thentheresultholds.Hence,assumethat and. Becausecontains no isolated $K_{1},\,0u$ has a neighborin. Ifor , then the result holds. Therefore, assume that and. Becausecontains no isolated $K_{2},$  eitherorhas a neighborin. If or $(0w,1\overline{{w}})\notin F_{c},$  then the result holds. Hence, assumethatand.Because contains no isolated $P_{3},~~0u$ ororhas a neighborin. Ifor, then the result holds. Therefore, assume thatand

          

         

        

       

      

     

    

   

  

 



. For convenience, let

,and

◽.

Claim 1. There are in totalpairwise edge-disjoint paths in which each connectsto a vertex inunder.



Proof of the claim. Thesepathsare constructed byexaminingeach vertex. By using Lemma 11, one or two neighboring edges ofare incident to. If $e$ is the unique neighboring edge of incident to $0y,$  chooseas the path for $e$ . Otherwise, letandbe two neighboring edges ofincident to. In this case, we chooseandas the paths for and $e^{\prime\prime},$  respectively. By examining each $0y,$  construct

     

    

   

  

 



be connected to a vertex inby using the bold edges. Fig. 7. Illustration of the proof of Case 3 in Theorem 2.

pairwise edge-disjoint paths where each connectsto a vertex in under.



One can perform the rest of proof in this way. By using Claim 1, there are(by using Lemmas 1-3) pairwise edge-disjoint paths in which each connectsto a vertex in under. However, these paths contain at most faulty edges. Hence,can be connected to a vertex inunder. ◽ Theorem 2.for.

     

    

   

  

 



Proof.foris demonstrated by ﬁnding an -cut of size $\mathcal{R}_{3}^{\prime}$ in. Letbe a cycle of length 4 in.



The edge-neighborhood of(i.e.,) is the desired -cut as follows. According to Lemma 4, $\mathcal{R}_{3}^{\prime}$ .



Let. Becauseand for $n\geq4,$ is connected. Furthermore, for. Hence,is the desired -cut implying $\mathcal{R}_{3}^{\prime}.$ for.

  

 



Next, we show thatfor. In other words, the size of every  $\mathcal{R}_{3}^{\prime}$ -cut ofis at least. For the sake of contradiction, suppose thatis a subset of edges inwith size , andcontains no isolated $K_{1},$  isolated , or isolated. Showing thatis connected yields a contradiction.

     

    

   

  

 



Letfor eachbe the set of edges in in which the two end-vertices differ in the th-bit position. Note that $P M_{n},$  and form a partition of $E(F Q_{n+1}).\operatorname{If}|P M_{i}\cap F|\leq2$ for every and $|{\overline{{P M}}}\cap F|\leq2,$  thenfor.

   

  

 



Hence,orfor some. In addition, the vertices ofcan be labeled such that



.

Let, $F_{1}=F\cap Q_{n}^{1},$ and

.Therefore,and

. Without loss of generality, is assumed. Thus, we have $2|F_{1}|\leq|F_{0}|+|F_{1}|\leq|F|\leq4n-4,$  which leads to. By using Lemma 12, then satisﬁes one of the following three scenarios: (1)is connected; (2)has two components, one of which is an isolated vertex; and (3)has two compo- nents, one of which is an isolated edge.

     

    

   

  

 



Case 1.is connected. Because every vertex inis connected to a vertex in(by using Lemma 13), is connected.

 



Case 2.has two components, one of which is an isolated vertex; that is,. It remains to show thatcan be connected to in. Note that . Becausecontains no isolated $K_{1},\,(0t,1t)\notin F_{c}$ or

  

 



. This leads to the following scenarios.

### Case2.1.and.Ifor

, then the result holds. Hence, assume that and(seeFig.6a).Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $(0u,1\overline{{u}})\notin F_{c},$  then the result holds. Therefore, assume thatand

   

  

 



.Forconvenience,let and. According to an argu- mentsimilartoClaim $^{1,}$ thereareintotal pairwise edge-disjoint paths where each connectstounder . However, these paths containatmost faulty edges. Hence,can be connected to a vertex inunder.

       

      

     

    

   

  

 



## Case 2.2.and $(0\bar{t},1t)\in F_{c})$  or  $((0t,1t)\in F_{c}$ and

).Withoutlossofgenerality,assumethat and(see Fig. 6b). If $(0t,1\bar{t})\not\in\,F_{c},$  then the result holds. Hence, assume that. Because contains no isolated $K_{2},$ has a neighborin . Ifor $(0u,1\overline{{u}})\notin F_{\mathrm{c}},$  then the result holds.

   

  

 



Hence, assume thatand. Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $(0v,1\overline{{v}})\notin F_{c},$  then the result holds. Therefore, assume thatand

  

 



. For convenience, let and. Ac- cording to an argument similar to Claim 1, there are in total pairwise edge-disjoint paths in which each connectsto under. However, these paths contain at most faulty edges. Hence,can be connected to a vertex in under.

      

     

    

   

  

 



Case 3.has two components, one of which is an isolated edge; that is,  $(1u,1v)$ ). It remains to show that  $(1u,1v)$ ) can be connected tounder. Because

 



 $\left|F\right|=4n-1,\,\left|F_{c}\right|\geq3,\,\left|F_{0}\right|\geq\left|F_{1}\right|$ ,and , it leads toand. Consider a setofpairwiseedge-disjointpaths

 



, where each path con- nects $(1u,1v)$  $(1u,1v)\,{\mathrm{to}}\,Q_{n}^{1}-F_{1}\,-\{1u,1v\}$ (seeFig.7).Because(1)each edge inbelongs to $E_{\mathrm{c}},$  (2) $|E D P|=4,$  and (3) $|F_{c}|=3,$  a fault-free path exists in. Hence,  $(1u,1v)$ ) can be connected tounder. ◽

   

  

 


# On 3-Extra Connectivity and 3-Extra Edge

# Connectivity of Folded Hypercubes

Nai-Wen Chang, Cheng-Yen Tsai, and Sun-Yuan Hsieh, Senior Member, IEEE

Abstract—Given a graphand a non-negative integer $g,$ , the $g_{\parallel}$ -extra connectivity (resp. $g_{\parallel}$ -extra edge connectivity) ofis the minimum cardinality of a set of vertices (resp. edges) in, if it exists, whose deletion disconnectsand leaves each remaining component with more thanvertices. This study shows that the 3-extra connectivity (resp. 3-extra edge connectivity) of an-dimensional folded hypercube isfor(resp.for). This study also provides an upper bound for the $g_{\parallel}$ -extra connectivity on folded hypercubes for.

     

    

   

  

 



Index Terms—Interconnected networks, connectivity, extra connectivity, extra edge connectivity, fault-tolerance, reliability

1INTRODUCTION BECAUSE of the rapid development of VLSI technology, a multipro- cessor system may contain hundreds or even thousands of nodes, some of which may be faulty when the system is implemented. As the number of processors in a system increases, the possibility that its processors may become faulty also increases. Because designing such systems without defects is nearly impossible, and fault tolerance are two of the most critical concerns of multiprocessor systems [19].



Based on the deﬁnition proposed by Esfahanian [1], a multi- processor system is fault tolerant if it can remain in the presence of failures. Two basic functionality criteria have received considerable attention. The ﬁrst criterion for a system to be re- garded as functional is whether the network logically contains a certain topological structure. This is the problem that occurs when embedding one architecture into another [11], [18]. This approach involves using system-wide redundancy and reconﬁguration. The second functionality criterion considers a multiprocessor system functional if a fault-free communication path1 exists between any two fault-free nodes; that is, the topological structure of the multiprocessor system remains connected in the presence of cer- tain failures. Thus, and edge connectivity are two major measurements of this criterion [18]. The connectivity of a graph $G,$  denoted by $\kappa(G),$  is the minimal number of vertices whose removal fromproduces a disconnected graph or only one vertex; the edge connectivity of a graph $G,$  denoted by $\lambda(G),$  is the minimal number of edges whose removal fromproduces a disconnected graph.

    

   

  

 



However, these two parameters tacitly assume that all vertices that are adjacent to, or all edges that are incident to, the same vertex can potentially fail simultaneously. This is practically impossible in some network applications. To address this deﬁciency, two speciﬁc terms forbidden faulty set and forbidden faulty edge set are introduced. The vertices in a forbidden faulty set or the edges in a forbidden faulty edge set cannot fail simultaneously. By restricting the for- bidden faulty set (resp. forbidden faulty edge set) to the sets of neighbors (resp. neighboring edges) of any spanning subgraph with no more than $g$ -vertices in the faulty networks, Fàbrega and Fiol [5], [6] introduced the following measures. Given a graphand a non-negative integer $g,$  the $g$ -extra connectivity (resp. $g_{-}$ -extra edge connectivity) of $G,$  written as(resp. $\lambda_{g}(G)),$  is the minimal cardinality of a set of vertices (resp. edges) in $G,$  if it exists, whose deletion disconnectsand leaves each remaining component with more thanvertices.

    

   

  

 



Note thatandfor any graphifis not a complete graph. Therefore, the-extra connectivity (resp.



-extra edge connectivity) can be regarded as a general form of the classical connectivity (resp. classical edge connectivity) that pro- vides measures that are more accurate for reliability and fault- tolerance for large-scale parallel processing systems. Regarding the computational complexity of the problem, based on thorough research, no polynomial-time algorithm has been presented to computefor a general graph; nor has there been any tight upper bound for[2]. However,can be computed by solving numer- ous network ﬂow problems [3]. The $g$ -extra connectivity and $g_{\l}$ -extra edge connectivity of particular classes of graphs have recently received considerable attention [2], [3], [5], [6], [10], [16], [17], [20]-[23], [25], [26].

 



The hypercube is one of the most recognized interconnected net-

### works. An-dimensional hypercubeis an-regular graph with

vertices andedges. Each vertexincan be labeled by an -bit binary string. Two vertices are joined by an edge if and only if their binary strings differ in a one-bit position only. The recursive deﬁnition ofis as follows:is a complete graph with two vertices labeled by 0 and 1. Letbe a “don’t care” symbol. For $n\geq2,$ is constructed from two copies of  $(n-1)\cdot$ )- dimensional hypercubes (subcubes), namelyand, where is a subgraph ofinduced by vertex subsetand is a subgraph ofinduced by vertex subset. Edges exist between corresponding vertices in the two subcubes such that vertexinis adjacent to vertexin. The folded hypercube is an attractive variation of the hypercube pro- posed by El-Amawy and Latiﬁ [1]. An-dimensional folded hypercube can be constructed from an $n$ -dimensional hypercube by adding edges, called “complementaryedges”, between opposite pairs of verticesand $\overline{{u}}=\overline{{u}}_{1}\overline{{u}}_{2}\ldots\overline{{u}}_{n},$ where $\overline{{u}}_{i}=1-u_{i}.F Q_{n}$ is an  $(n+1)$ )-regular  $(n+1)\cdot$ )-connected graph withvertices and edges. The folded hypercube not only preserves the advantages of the hypercube, such as its symmetric properties, but also possesses properties that are superior to those of the hypercube. The topological properties of folded hypercubes have been exten- sively investigated [1], [8], [12], [20], [25], [26]. Folded hypercubes have been used as underlying topologies of several parallel systems, such as ATM switches [13], [14], PM2I networks [9], and 3D-FolH- NOC networks [4], for high-speed cell-switching and reducing the diameter and trafﬁc congestion of the hypercube with little hardware overhead.

             

            

           

          

         

        

       

      

     

    

   

  

 



The probability that a vertex or an edge fails is called “failing probability”. For simplicity, assume that each vertex (resp. edge) possesses the same failing probability, denoted by $p\left(\operatorname{resp.}q\right)$ . Further, let(resp. $\xi_{1}^{\prime},\xi_{2}^{\prime},\xi_{3}^{\prime})$  denote the events that all of the neighbors (resp. neighboring edges) of any vertex, any edge, any path of 3 vertices infail simultaneously, respectively. Their probabilities aregivenby $p r o b(\xi_{1})\leq2^{n}\cdot p^{n+1},$ , ,, ,for. It can be seen that these probabilities become relatively small for sufﬁ ciently large. For example, by adoptingand $n=5,$  theprobabilityofisolatingacomponentwithatmost3 vertices is calculated as

        

       

      

     

    

   

  

 



, which is rela tively small. Therefore, the probability of isolating a component with at most three vertices is extremely small (i.e., the probabilities of all of the neighbors (neighboring edges) of any vertex, any edge, or any path of 3 vertices failing simultaneously inis extremely small).



Therefore, consideringandis reasonable and meaningful.Previousresultshaveshownthat for[1],for [1],for[20],for [25],for[26], andfor [26]. This paper shows thatforand for. This paper also provides an upper bound offor.

      

     

    

   

  

 



The remainder of this paper is organized as follows. Section 2 introduces necessary deﬁnitions and notations. Section 3 provides basic properties related to bothand. Section 4 explores the 3-extra connectivity and 3-extra edge connectivity of. Finally, Section 5 presents the conclusion of the paper.

  

 



2PRELIMINARIES A graphis a pair of the vertex setand the edge set $E,$  whereisaﬁnitesetandisasubsetof

 



. Useandto denote the vertex set and edge set of, respectively. This study considers undirected, simple, and connected graphs. Two verticesandare

 



## adjacent if $(u,v)\in E(G),$  andandare end-vertices of  $(u,v)$ ). A

subgraphof, denoted by $H\subseteq G,$  is a graph whereand. The components ofare its maximal connected subgraphs.

 



For any vertex $u\operatorname{in}G,$  $G,$  the neighborhood(neighbors) of $u\operatorname{in}G,$ , denoted by, is the set of vertices adjacent toin. The edge neighborhood (neighboring edges) ofin $G,$  denoted by $N E_{G}(u),$  is the set of edges incident toin. The degree of a vertexin $G,$  denoted by $d e g_{G}(u),$  is thecardinality of. Agraphis -regular ifforevery vertexin. Letbe any vertex subset of; that is,. The

    

   

  

 



### neighborhood and edge-neighborhood ofinare deﬁned as

### , respectively. The closed neighborhood ofinis

deﬁned as. For a subgraphin, the notation (resp.) can be simpliﬁed as(resp.



). The notationcan be simpliﬁed as.

A pathforin a graphis a sequence of distinct vertices such that any two consecutive vertices are adjacent. A cycleforis a sequence of vertices in which any two consecutive vertices are adjacent, where are all distinct. The girth ofis the length of the shortest cycle in. A complete graphis a graph consisting ofpairwise adjacent vertices.

  

 



A complete bipartite graphis a graph consisting of two vertex setsand $V_{\mathrm{{2}}},$  called partite sets, with ordersand $s,$  respectively. Two vertices are adjacent if and only if they are in different partite sets (i.e., for every pairand $y\in V_{2},$  an edge  $(x,y)$ ) is in). Useto denote the distance of two verticesand

   

  

 



### in.Thediameterofisdeﬁnedas

. An isomorphism from a simple graph to a simple graphis a one-to-one and an onto function suchthatifandonlyif

 



## . Note that  $^{\prime\prime}G$ is isomorphic to written as $H,^{\prime\prime}$ 

, if there is an isomorphism fromto. Letbe a non- negative integer and  $\mathcal{P}_{g}$ be the property of having more than vertices. Given a vertex subset (resp. edge subset)in a graph

 



, use the notationto represent the subgraph of obtained after deleting all vertices (resp. edges) infrom. For convenience, refer to the vertices (edges) inas faulty vertices (resp.

 



faulty edges). A faulty vertex subsetis an  $\mathcal{R}_{g}$ -cut ifis disconnected and every remaining component ofhas property . A faulty edge subset $\mathcal{P}_{g}$ is a (- $\mathcal{R}_{g}^{\prime}.$  cut for short) ifis disconnected and every remaining component of

  

 



## has property  $\mathcal{P}_{g}$ . The-extra connectivity of $G,$  denoted by

, equals $\mathcal{R}_{g}$ . Similarly, the $g$ -extra edge

## connectivity of $G,$  denoted by $\lambda_{g}(G),$  equals

R.

 $\mathrm{An}\,(n+1)\!.$ )-dimensional folded hypercubecan be regarded as a graph consisting of two-dimensional hypercubesand $Q_{n}^{1},$  denoted by $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  whereandare two-dimen- sional hypercubes with the preﬁx bits 0 and 1 in each vertex, respectively.Thesymbolrepresentstheperfectlymatched operation that connectsandby using two speciﬁc perfect matchingsand

    

   

  

 



. For convenience, also use the notationto represent. In addition, use  $0u\,(\mathrm{resp.}\,1u)$ ) to represent a vertex ofbelonging to(resp.

 



). Any vertexhas two neighbors and any vertexhas two neighbors. The girth ofequals 4 for[26].

 



For any two verticesand $y,$  useto denote the Hamming distance betweenand $y;$  that is, the number of different positions between the binary strings ofand $y$ . A summary of notations and symbols is listed in Table 1.

  

 



3BASIC PROPERTIES Lemmas 1-3 can be easily veriﬁed becauseis-regular and triangle-free for every.

 



### Lemma1. If()admitsasasubgraph,then

.

## Lemma2. If $(n\geq3)$ )admitsasasubgraph,then

### if the end-vertices ofare adjacent, and

otherwise.

## Lemma3. If $(n\geq3)$ )admitsasasubgraph,then

.

is ()-regular and triangle-free forbecause the girth equals 4; therefore, the following lemma is derived.



## Lemma 4. Letbe a subgraph of $F Q_{n+1},$  where. Then,

.

### Lemma 5. [22] Letbe any subset of the vertices ofwith

, where. Then,.

## Lemma 6. [22] Letbe a subgraph of $Q_{n},$  where

### and.Then,thefollowingtwostatementshold:

(1). (2)is connected with property  $\mathcal{P}_{g}$ .



## Lemma 7.  $\it{122}J\it{L e t}V^{\prime}$ be a vertex subset inwith $|V^{\prime}|=g+1,$  where

### and. Then,is connected with

property  $\mathcal{P}_{g}$ .

## Lemma 8. Let $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  where $n\geq5,$  and letbe a

### connectedsubgraphof.If $d i a m(A)\leq n-1,$ then

.

Proof. Letandbe any two vertices in. Ifandhave common neighbors in $Q_{n}^{1},$  thenor $1\overline{{u}}=1v\;(\mathrm{i.e.,}\,0u=0\overline{{v}}),$ ), whichleadstoacontradictionbecause

 



. In other words, any two distinct vertices inhave no common neighbors in.



Furthermore, because each vertex inhas two neighbors in $Q_{n}^{1},$  . ◽



## Lemma 9. Let $F Q_{n+1}=Q_{n}^{0}\otimes Q_{n}^{1},$  where $n\geq5,$  and letbe a

### subgraphof $Q_{n}^{0},$ where.Then,thefollowingtwo

statementshold:(1).

(2)is connected with property  $\mathcal{P}_{g}$ .

Proof. By using Lemmas 6 and 8, Thus, Condition 1 holds.



By using Lemmas 6 and 7 (by lettingin Lemma 7), bothandareconnected.



Because forand,is connected to , which leads tobeing con- nected. In addition, because forand, has property  $\mathcal{P}_{g}$ . ◽ 4MAIN RESULTS

     

    

   

  

 



## 4.13-Extra Connectivity

## Lemma10. Let,andlet $F\subset V(F Q_{n+1}),$ 

## , and $F_{1}=F\cap V(Q_{n}^{1}),$  where. If

## anddoes not contain isolated $K_{1},$  isolated $K_{2},$  or

## isolated $P_{3},$  then every vertex inis connected to  $a$  vertex in

under.

Proof. Letbe any vertex in(see Fig. 1). Ifor , then the result holds. Hence, assume thatand . Becausecontains no isolated $K_{1},$ has a neighborin. Ifor $1\overline{{v}}\notin F_{1},$  then the result holds.

  

 



Hence, assume thatand. Because contains no isolated $K_{2},$  eitherorhas a neighborin . Ifor $1\overline{{w}}\notin{\cal F}_{1},$  then the result holds. Hence, assume thatand. Becausecontains no isolated $P_{3},\{0u,0v,0w\}$ has a neighborin. If or $1{\overline{{x}}}\notin F_{1},$  then the result holds. Therefore, assume that and. For convenience, letand

     

    

   

  

 



.

Letbe the connected subgraph induced by $V^{\prime}\operatorname{in}Q_{n}^{0}$ . Because and $d i a m(A^{\prime})\leq3,$  all vertices inare distinct (i.e., ). Let. Letbe any vertex inand letbe the connected subgraph induced by (in $Q_{n}^{0})$ . Becauseand $d i a m(A^{\prime\prime})\leq4,\ 1y\notin{\cal F}^{\prime},$ , implying that only vertices incan be the faulty vertices in. Because(by using Lemma 5) and contains at mostfaulty verti- ces, an edgeexists such that (1)(i.e., ), and (2)(i.e., $1y\notin F_{1}-F^{\prime}),$  implying that because. Therefore,can be connected to a vertex in. ◽ Theorem 1.for.

           

          

         

        

       

      

     

    

   

  

 



Proof.forisdemonstratedby constructing an  $\mathcal{R}_{3}$ -cut of sizeinas follows. Let , and letbe a connected subgraph of.

 



Using Lemma 9 then yields (1)

, and (2)is connected with property  $\mathcal{P}_{3}$ . Therefore,is the desired  $\mathcal{R}_{3}$ -cut, implying thatfor.

 



Next, provethatfor. In other words, the size of every  $\mathcal{R}_{3}$ -cut ofis at least. For the sake of contradiction, supposewithis an -cut; that is, $\mathcal{R}_{3}$ does not contain isolated $K_{1},$  isolated , or isolated. Showing thatis connected yields a contradiction.

    

   

  

 



Letand. Without loss of generality, assume that. Thus,



, implyingfor.

According to the result in [7],then satisﬁes one of the following four scenarios: (1)is connected, (2) contains two components, one of which one is an isolated vertex, (3)contains two components, one of which is an isolated edge. (4)contains three components, two of which are isolated vertices.

   

  

 



Case 1.is connected. Because every vertex inis connected to a vertex in(by using Lemma 10), is connected.

 



Case 2.contains two components, one of which is an isolated vertex; that is,. It remains to show thatcan be connected toby using some vertices in .Becausecontainsnoisolated $K_{1},$  or. If $1\bar{t}\notin F_{1},$  then the result holds. Hence, assume that. Consider the following two subcases.

    

   

  

 



Case 2.1.and. Becausecontains no isolated $P_{3},$  eitherorhas a neighborin. If or $1{\overline{{u}}}\not\in F_{1},$  then the result holds. Hence, assume that and. Without loss of generality, assume that is adjacent to. For the neighbors ofin $Q_{n}^{0},$  the following two scenarios are considered further: Case 2.1.1 $N_{Q_{n}^{0}}(0\bar{t})\not\subset F_{0}$ . Letbe a vertex in (seeFig.2a).Ifor $1{\overline{{v}}}\notin F_{1},$ thentheresult holds. Hence, assume thatand. Because ,, imply- ing

         

        

       

      

     

    

   

  

 



. Clearly,. Because andcontainsatmost faulty vertices, an edgeexists such thatand.

  

 



## Case 2.1.2. Because $d_{H}(0t,0\bar{t})=d_{H}(1t,1\bar{t})=n\geq5,$ 

and

. Therefore, because andcontainsatmost faulty neighbors, eitherorhas a fault-free neighborin (seeFig.2b).Let

   

  

 



.Clearly,.

Because(by using Lemma 5) andcontains at most faulty vertices, an edge exists such thatand.

  

 



Case 2.2. (and) or (and $0\Bar{t}\not\in F_{0})$ . Without loss of generality, assume thatand. If $1{\bar{t}}\not\in F_{1},$  then the result holds. Hence, assume that. Because contains no isolated $K_{2},\,0t$ has a neighborin (see Fig. 3). Ifor $1{\overline{{u}}}\notin F_{1},$  then the result holds.

   

  

 



Therefore,assumethatand.Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $1{\overline{{v}}}\notin F_{1},$  then the result holds. Hence, assume thatand. Because ,and , implying and .Because(byusing Lemma 5) andcontains at most faulty neighbors in $Q_{n}^{0},$ has a neighborin. Ifor $1\overline{{w}}\notin F_{1},$  then the result holds. Therefore, assume thatand.

          

         

        

       

      

     

    

   

  

 



Forconvenience, letand

. The remaining proof is similar to the method described in Paragraph 2 in the proof of Lemma 10.



Case 3.contains two components, one of which is an isolated edge; that is,  $(1u,1v)$ ). Furthermore,  $(1u,1v)$ ) can be connected toby using some vertices in

 



. Becauseand

,or. First consider the case that and. Becausecontains no isolated  $K_{2},\ \left|\{0u,0\overline{{{u}}},0v,0\overline{{{v}}}\}\cap F_{0}\right|\neq4$ . Hence,  $(1u,1v)$ ) can be con- nected toby using vertices inand the result holds. Next, consider the case thatand (orand). Ifor $0{\overline{{v}}}\notin F_{0},$  then the result holds. Hence, assume thatand. Because contains no isolated,or.

      

     

    

   

  

 



Notethatbecauseand (because),



. Furthermore, because and $|F_{0}|\geq|F_{1}|=2n-1,|F_{0}|=|F|-|F_{1}|=2n-1.$ . This leads to the following scenarios.

 



can be connected to a vertex inby using the bold edges.

inand, at most one isolated vertex exists in . Conversely, because at leastedges must be removed to obtain an isolated vertex subsetin, where , andfor,satisﬁes one adjacent to. Thus,is also adjacent to. Because is adjacent toand $d_{H}(1\overline{{u}},1u)\geq5,$ , which leads tobecause. Therefore,  $(1u,1v)$ ) can be connected to a vertex inby using and.

       

      

     

    

   

  

 



### Case 3.2. (and) or (and). With-

out loss of generality, assume thatand. Recall that. Becausecontains no isolated $P_{\mathrm{{3}}},$ has a neighborin(see Fig. 4b). The remainder of the proof is similar to that used in Case 3.1.

  

 



Case 4.contains three components, two of which are isolated vertices; that is,and. Based on the method similar to that used in Case 3, we can show thatand can be connected toby using some vertices in.

   

  

 



Combining the above cases completes the proof. ◽ Based on Lemma 9 and the proof of Theorem 1, the following corollary extends the results to suggest an upper bound of.

 



## Corollary 1., whereand

.

## 4.23-Edge Extra Connectivity

## Lemma 11. Letbe a connected subgraph ofwith $|V(A)|=4,$ 

### where. Then, each neighbor ofinis adjacent to at most two

vertices in.

Proof. Note thatcan be isomorphic to $P_{4,}$ or. Letbe a neighbor ofin. Suppose, conversely, thatis adjacent to at least three vertices in. Hence, either (1) a triangle is generated, or (2) two vertices having at least three common neighbors exist in , which leads to a contradiction. ◽ Lemma 12. Letwith $|F|\leq2n-2,$  where. Then,

    

   

  

 



### satisﬁes one of the following three scenarios: (1)

### is connected. (2)has two components, one of which is an

### isolated vertex. (3)has two components, one of which is an

isolated edge.

Proof. Ifis connected, then Condition 1 holds. Otherwise, assume thatis disconnected. Because



[25], where,contains at least one isolated vertex or one isolated edge. In addition, because at of the following two scenarios: (i) There is exactly one isolated vertex (hereafter, denoted by ) in. (ii) There is exactly one isolated edge, hereafter denoted by  $(u,v),$ ), in.

  

 



If (i) holds, thenis connected as follows.

Becauseand ,is connected. Hence, Condition 2 holds. If (ii) holds, then is connected as follows. Because forand ,is connected. Hence, Condition 3 holds. ◽

     

    

   

  

 



## Lemma13. Let.Let

### ,,,and

### , where. Suppose thatand

## does not contain any isolated $K_{1},$  isolated $K_{2},$  or

## isolated. Then, every vertex inis connected to  $a$  vertex in

under.

Proof. Letbe any vertex in(see Fig. 5). Ifor ,thentheresultholds.Hence,assumethat and. Becausecontains no isolated $K_{1},\,0u$ has a neighborin. Ifor , then the result holds. Therefore, assume that and. Becausecontains no isolated $K_{2},$  eitherorhas a neighborin. If or $(0w,1\overline{{w}})\notin F_{c},$  then the result holds. Hence, assumethatand.Because contains no isolated $P_{3},~~0u$ ororhas a neighborin. Ifor, then the result holds. Therefore, assume thatand

          

         

        

       

      

     

    

   

  

 



. For convenience, let

,and

◽.

Claim 1. There are in totalpairwise edge-disjoint paths in which each connectsto a vertex inunder.



Proof of the claim. Thesepathsare constructed byexaminingeach vertex. By using Lemma 11, one or two neighboring edges ofare incident to. If $e$ is the unique neighboring edge of incident to $0y,$  chooseas the path for $e$ . Otherwise, letandbe two neighboring edges ofincident to. In this case, we chooseandas the paths for and $e^{\prime\prime},$  respectively. By examining each $0y,$  construct

     

    

   

  

 



be connected to a vertex inby using the bold edges. Fig. 7. Illustration of the proof of Case 3 in Theorem 2.

pairwise edge-disjoint paths where each connectsto a vertex in under.



One can perform the rest of proof in this way. By using Claim 1, there are(by using Lemmas 1-3) pairwise edge-disjoint paths in which each connectsto a vertex in under. However, these paths contain at most faulty edges. Hence,can be connected to a vertex inunder. ◽ Theorem 2.for.

     

    

   

  

 



Proof.foris demonstrated by ﬁnding an -cut of size $\mathcal{R}_{3}^{\prime}$ in. Letbe a cycle of length 4 in.



The edge-neighborhood of(i.e.,) is the desired -cut as follows. According to Lemma 4, $\mathcal{R}_{3}^{\prime}$ .



Let. Becauseand for $n\geq4,$ is connected. Furthermore, for. Hence,is the desired -cut implying $\mathcal{R}_{3}^{\prime}.$ for.

  

 



Next, we show thatfor. In other words, the size of every  $\mathcal{R}_{3}^{\prime}$ -cut ofis at least. For the sake of contradiction, suppose thatis a subset of edges inwith size , andcontains no isolated $K_{1},$  isolated , or isolated. Showing thatis connected yields a contradiction.

     

    

   

  

 



Letfor eachbe the set of edges in in which the two end-vertices differ in the th-bit position. Note that $P M_{n},$  and form a partition of $E(F Q_{n+1}).\operatorname{If}|P M_{i}\cap F|\leq2$ for every and $|{\overline{{P M}}}\cap F|\leq2,$  thenfor.

   

  

 



Hence,orfor some. In addition, the vertices ofcan be labeled such that



.

Let, $F_{1}=F\cap Q_{n}^{1},$ and

.Therefore,and

. Without loss of generality, is assumed. Thus, we have $2|F_{1}|\leq|F_{0}|+|F_{1}|\leq|F|\leq4n-4,$  which leads to. By using Lemma 12, then satisﬁes one of the following three scenarios: (1)is connected; (2)has two components, one of which is an isolated vertex; and (3)has two compo- nents, one of which is an isolated edge.

     

    

   

  

 



Case 1.is connected. Because every vertex inis connected to a vertex in(by using Lemma 13), is connected.

 



Case 2.has two components, one of which is an isolated vertex; that is,. It remains to show thatcan be connected to in. Note that . Becausecontains no isolated $K_{1},\,(0t,1t)\notin F_{c}$ or

  

 



. This leads to the following scenarios.

### Case2.1.and.Ifor

, then the result holds. Hence, assume that and(seeFig.6a).Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $(0u,1\overline{{u}})\notin F_{c},$  then the result holds. Therefore, assume thatand

   

  

 



.Forconvenience,let and. According to an argu- mentsimilartoClaim $^{1,}$ thereareintotal pairwise edge-disjoint paths where each connectstounder . However, these paths containatmost faulty edges. Hence,can be connected to a vertex inunder.

       

      

     

    

   

  

 



## Case 2.2.and $(0\bar{t},1t)\in F_{c})$  or  $((0t,1t)\in F_{c}$ and

).Withoutlossofgenerality,assumethat and(see Fig. 6b). If $(0t,1\bar{t})\not\in\,F_{c},$  then the result holds. Hence, assume that. Because contains no isolated $K_{2},$ has a neighborin . Ifor $(0u,1\overline{{u}})\notin F_{\mathrm{c}},$  then the result holds.

   

  

 



Hence, assume thatand. Because contains no isolated $P_{3},$  eitherorhas a neighborin. Ifor $(0v,1\overline{{v}})\notin F_{c},$  then the result holds. Therefore, assume thatand

  

 



. For convenience, let and. Ac- cording to an argument similar to Claim 1, there are in total pairwise edge-disjoint paths in which each connectsto under. However, these paths contain at most faulty edges. Hence,can be connected to a vertex in under.

      

     

    

   

  

 



Case 3.has two components, one of which is an isolated edge; that is,  $(1u,1v)$ ). It remains to show that  $(1u,1v)$ ) can be connected tounder. Because

 



 $\left|F\right|=4n-1,\,\left|F_{c}\right|\geq3,\,\left|F_{0}\right|\geq\left|F_{1}\right|$ ,and , it leads toand. Consider a setofpairwiseedge-disjointpaths

 



, where each path con- nects $(1u,1v)$  $(1u,1v)\,{\mathrm{to}}\,Q_{n}^{1}-F_{1}\,-\{1u,1v\}$ (seeFig.7).Because(1)each edge inbelongs to $E_{\mathrm{c}},$  (2) $|E D P|=4,$  and (3) $|F_{c}|=3,$  a fault-free path exists in. Hence,  $(1u,1v)$ ) can be connected tounder. ◽

   

  

 


# Effects of placebo on bench throw performance of Paralympic weightlifting athletes: a pilot study

Gustavo De Conti Teixeira Costa, Luan Galvão, Martim Bottaro, João Felipe Mota, Gustavo Duarte Pimentel and Paulo Gentil

## Abstract

Background: The aim of the present study was to analyse the effects of placebo on bench throw performance in Paralympic weightlifting athletes.

Methods: The study involved four Paralympic weightlifting male athletes (age:  $40.25\pm9.91$  years, weight:  $60.5\pm8.29\,\mathsf{k g},$  height:  $1.60\pm0.15\,\upmu\mathrm{m})$  that visited the laboratory in three occasions, separated by  $72\,\mathrm{h}$ . In the first session, the athletes were tested for bench press one repetition maximum (1RM). The other two sessions were performed in a randomized counter-balanced order and involved bench throw tests performed either after taking placebo while being informed that the capsule contained caffeine or without taking any substance (control). The bench throw tests were performed with loads corresponding to 50, 60, 70 and  $80\%$  of the bench press 1RM.

Results: According to the results, mean velocity  $(\Delta;0.08\,\mathsf{m}/\mathsf{s},\mathsf{E S}\,0.36,p<0.05)$ and mean propulsive velocity  at  $50\%$  of 1RM were significantly higher during placebo than control  $(p<0.05)$  $(\Delta;0.1\;1\;\mathsf{m}/\mathsf{s},\mathsf{E S}\;0.49,p<0.05)$ .

However, there were no difference between control and placebo for 60, 70 and  $80\%$  of 1RM  $(p>0.05)$ .

Conclusion: Our results suggest that placebo intake, when the athletes were informed they were taking caffeine, might be an efficient strategy to improve the performance of explosive movements in Paralympic weightlifting athletes when using low-loads. This brings the possibility of using placebo in order to increase performance, which might reduce the risks associated with ergogenic aids, such as side-effects and positive doping testing.

Keywords: Nutritional supplements, Sports performance, Psyching up

Background The intake of nutritional supplements are a common practiceamongphysicalactivitypractitionersand athletes; however, few actually have scientific evidence for their efficacy [1]. Among them, caffeine is one of the most consumed substances by athletes [2] and is consid- ered to positively impact physical performance [3]. In this regard, previous studies suggested that caffeine may be ergogenic, sparing muscle glycogen improving pain tolerance, reducing rate of perceived exertion, increase maximum voluntary contraction, strength and power in high-intensity activities besides of stimulating central nervous system [4–10]. However, there are specific controversies about its effects, mainly when the studies compared the acute effects of caffeine vs placebo intake showed inconsistent conclusions [11–17].



While genetic factors might explain a large portion of the variance associated with the caffeine effects such as pain tolerance, anxiogenic and alert effects [18–20], there are important psychological responses to ingesting a substance that should be considered. The placebo effect [21] can influence the physiological aspects to physical exercise performance [3], one of the all factors that might influence the effects of caffeine is the placebo effect [22]. In agreement with this, Saunders et al. [23] found an improvement in the cycling time to exhaustion in trained cyclists who ingested placebo believing to have ingested caffeine. Similarly, in a previous study, Beedie et al. [24] reported that when competitive male cyclists ingested placebo believing to have ingested caf- feine there were increases in aerobic power at VO2max test and  $10\mathrm{-km}$  time trials, with no difference in oxygen uptake, heart rate, and blood lactate. Such placebo ef- fects do not rule out a true effect of caffeine supplemen- tation. Although these studies suggested that aerobic performance improves following placebo intake when in- dividuals believed that they are ingesting caffeine, we are not aware of studies that measured the placebo effect in powerlifting. Moreover, Paralympic athletes have been shown to have psychological particularities that might make them especially vulnerable to the placebo effect, such as, concerns about having to perform consistently well throughout training and difficulties in coping with negative results [25]. Therefore, it seems important to perform specific studies in these group despite the diffi- culty to select subjects with these characteristics.



The analysis of the placebo effect might be of great practical importance since it could provide an alternative for improving performance through the intake of an inert substance, with no risk of testing positive for dop- ing or adverse effects, either due to the direct use of pro- hibited substances, or by the possible contamination of nutritional supplements [10, 26–30]. Based on this, the present study aimed to evaluate if muscle performance during explosive movements would change in Paralympic weightlifting athletes after the intake of placebo when the participants were informed that they were taking caffeine.

Materials and methods Participants FourParalympicweightliftingmaleathleteswere recruited to participate in this study (age:  $40.25\pm9.91~\mathrm{y},$  weight:  $60.5\pm8.29\,\mathrm{kg},$  height:  $1.60\pm0.15\,\mathrm{m})$ . One had dwarfism, one myelomeningocele and hydrocephalus and two poliomyelitis. Athletes trained regularly five times a week aiming to compete, and all had previous experience with caffeine use, but have not taken any caffeine supplements in the previous six months. Two athletes had won medals in at least two phases on the national circuit. They were only allowed to participate if they had no orthopaedic or cardiometabolic problems that could be aggravated by the study protocol, as attested by a physician. This study was approved by Federal University of Goias committee (2.058.322) and all the participants signed a written informed consent form before participation (Table 1).

Procedures The study is a randomized, double-blind, crossover study. The athletes attended the laboratory three times, with an interval of  $72\,\mathrm{h}$  between visits. During the first visit,theyweresubmittedtotheonerepetition maximum test (1RM) in the bench press, as previously recommended [31]. The second and third visits involved the bench throw tests. The athletes were randomly assigned to ingest a capsule of placebo or no capsule on the second and third visits, in a cross-over design. During the placebo situations, the athletes received one capsule containing maize starch one-hour prior the test and were informed that it contained  $6\,\mathrm{mg.kg-1}$  caffeine. The athletes were oriented to avoid caffeine containing beverages and foods one week prior to the beginning of the study.



Data collection Muscular performance was measured in the bench throw, using an isoinertial indicator (T-Force, Dynamic Measurement System; Ergotech Consulting S.L., Murcia, Spain). The exercise was performed on a smith machine and the athletes were instructed to perform three repeti- tions with maximum intended velocity in all repetitions. The tests were performed with 50, 60, 70 and  $80\%$  of 1RM, with  $5\,\mathrm{min}$  of rest between each load condition. The movement started with elbows fully extended and then the bar was get down until touching the sternum. A linear position transducer was attached to the bar. The bar position data were sampled at  $1000\,\mathrm{Hz}$  using a computer, as recommended by the manufacturer. The finite differentiation technique was used to calculate the velocity and acceleration of the bar, presenting an associated error of  $<0.25\%,$  while the displacement was accurate to  $\pm0.5\:\mathrm{mm}$  [32].

Statistical analysis Data were analysed using the Statistical Package of Social Science software (SPSS 20.0, Chicago, IL, USA). Factorial ANOVA with a within-within design was used to compare the performance between placebo vs control situation at different loads. When necessary multiple comparisons were used as post hoc. Data were consid- ered statistically significant when  $p<0.05$ . Effect size (ES) of the mean differences was determined using Cohen’s d. The magnitude of the ES was determined by Hopkin’s scale as follows:  $<0.1$  (trivial), 0.1–0.3 (small), 0.3–0.5 (moderate), 0.5–0.7 (large), 0.7–0.9 (very large) and  $>0.9$  (perfect) [33].

Results Table 2 shows the results relative to the absolute velocity values as a function of the load lifted. Although the mean velocity to peak and peak velocity were not different be- tween situations, the mean velocity at  $50\%$  RM was signifi- cantly higher in placebo vs control  $(\Delta\colon\,0.08\,\mathrm{m}/s)$ , with moderate effect size  $(0.36;~~p<0.05)$ . Similarly, mean propulsive velocity at  $50\%$  RM was significantly higher in placebo vs. control  $(\Delta\colon0.11\,\mathrm{m}/s)$  with moderate effect size  as shown in Fig. $(0.49;p<0.05)$  1.

Discussion To the best of our knowledge, this was the first study to analyse the placebo effects of caffeine on bench throw performance of Paralympic weightlifting athletes. Accord- ing to the results, the ingestion of placebo significantly increases mean velocity and mean propulsive velocity at  $50\%$  of 1RM. These findings may be particularly interest- ing, since this intensity is in the recommend range for maximumpoweroutputinthebenchthrow[34]. Moreover, considering that the tests involved highly trained athletes, the differences might be relevant to training and competition. It is important to note that, whilst they did not reach significance, the differences with higher loads oc- curred in the opposite direction, with a trend for a detri- mental effect with placebo. The reason for this is not known, but there are two hypotheses to consider. First, the tests were incremental, so it might be possible that the higher performance in the earlier sets with lower loads lead to fatigue in the later sets, performed with higher loads. Second, when the load increased and becomes more chal- lenging, the participants might have expected to have an improved performance perform. However, since there was not physiologic enhancement due to the supplementation, a negative psychological influence might have occurred.

Although the deception used in the present study is not common in scientific literature, it is closer to what happens in real world, where many athletes take nutri- tional supplements believing on a true physiological effect, which might affect the results [22]. Besides that, uptake placebo was able to improve performance on bench throw with no reports of adverse effects already shown to caffeine supplementation before [10]. In this regard, Hurst et al. suggested that the intention to im- prove performance by the athletes when taking placebo can make a difference in the final performance; there- fore, in order to take full advantage of this intervention, the athletes should believe in the benefits of the ingested substance [35].



Whilst many athletes believe that nutritional supple- ments are related to performance enhancements [36], most do not obtain adequate information and do not even know the active ingredients or mechanism of actionofthesubstancesused[36–42].Therefore, supplements use seems to rely more on beliefs than on scientific evidence. Considering our findings that an inertsubstancemightincreaseperformancewhen athletes were deceived to believe it was an ergogenic aid; this might help to explain the divergence that often occurs between anecdotal and scientific evidence. On the other hand, this study had the focus only in placebo effect and a third group of caffeine was not used aiming to compare three groups.



Regarding the possible explanation for the placebo effect, Beedie et al. divided it in four categories: pain reduction, belief-behaviour relation, attentional changes and arousal changes. Within these mechanisms, the improvements found in our study can be explained by attentional and arousal changes [24]. Besides that, caf- feine intake also increasing pain tolerance [43], however, this probably did not happen in this study due to short duration with just three repetitions of exercise per- formed. The placebo effect might be associated with self-directed cognitive strategies and preparatory arousal (i.e. including imagery and attentional focus), which has been shown to enhance force production [44, 45].

The major limitation of the present study is the low number of participants. However, due to the characteristics of the participants, it would be difficult to obtain a higher sample size. It would also be interesting to have a third situation, with caffeine use. Future research should be con- ducted on a higher number of athletes, including Paralym- pic athletes with different limitations. Moreover, it would be valuable to assess the long-term effects of placebo, in order to test if the regular increase in performance over training sessions might bring long-term benefits.



Conclusions Our results suggest that placebo intake, when the athletes believe they are taking caffeine, might be an efficient strategy to improve performance in the bench throw test in Paralympic weightlifting athletes under low-loads. This brings the possibility of using placebo in order to increase performance in plyometric and speed exercises, reducing the side effects and risks associated with the use of ergogenic aids. Additionally, it would be ideal that nutritional strategy was investigated with high loads before using it in practice. Finally, it might be suggested that part of the conflict that usually exists be- tween anecdotal reports and scientific evidence about nutritional supplementation can be associated to the psychological effects of ingesting a supplement.

Acknowledgments Authors would like to thank to subjects that participated of this study for their commitment and effort.

Funding This research received no grant.

Availability of data and materials The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions GC participated in the study design, collected data, interpreted the data, and wrote the article. GC, LG, MB, JFM, GDP, and PG participated in the study design, and wrote the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate Participation in the study was voluntary, with written consent being obtained from each subject before the initiation of data collection. This study was conducted after review and approval by the Federal University of Goias committee. Committee’s reference number: 2.058.322.

Consent for publication All the study participants gave their consent to publish the research findings.

Competing interests The authors declare that they have no competing interests.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author details Faculdade de Educação Física e Dança, Universidade Federal de Goiás, Goiás, Brazil. Faculdade de Educação Física, Universidade de Brasília, Distrito Federal, Brasília, Brazil. Faculdade de Nutrição, Universidade Federal de Goiás, Goiás, Brazil.

### Received: 13 July 2018 Accepted: 12 February 2019

References 1.Maughan RJ, Burke LM, Dvorak J, Larson-Meyer DE, Peeling P, Phillips SM, et al. IOC consensus statement: dietary supplements and the high- performance athlete. Br. J. Sports Med. [Internet]. BMJ Publishing Group; 2018 [cited 2018 may 13];52:439–455. Available from: http://www.ncbi.nlm. nih.gov/pubmed/29540367.

2.Del Coso J, Muñoz G, Muñoz-Guerra J. Prevalence of caffeine use in elite athletes following its removal from the World Anti-Doping Agency list of banned substances. Appl. Physiol. Nutr. Metab. [Internet]. 2011;36:555–561. Available from: http://www.nrcresearchpress.com/doi/abs/10.1139/h11-052. 3.Lindheimer JB, O’Connor PJ, Dishman RK. Quantifying the Placebo Effect in Psychological Outcomes of Exercise Training: A Meta-Analysis of Randomized Trials. Sport. Med. 2015:693–711.

4.Lara B, Gonzalez-Millán C, Salinero JJ, Abian-Vicen J, Areces F, Barbero- Alvarez JC, et al. Caffeine-containing energy drink improves physical performance in female soccer players. Amino Acids. 2014;46:1385–92.

5.Fernandes-da-Silva J, Castagna C, Teixeira AS, Carminatti LJ, Guglielmo LGA. The peak velocity derived from the Carminatti test is related to physical match performance in young soccer players. J Sports Sci.

2016;34:2238–45.

6.Puente C, Abián-Vicén J, Salinero JJ, Lara B, Areces F, Del Coso J. Caffeine improves basketball performance in experienced basketball players.

Nutrients. 2017;9.

7.Goldstein E, Jacobs PL, Whitehurst M, Penhollow T, Antonio J. Caffeine enhances upper body strength in resistance-trained women. J Int Soc Sports Nutr. 2010;7.

8.Timmins TD, Saunders DH. Effect of caffeine ingestion on maximal voluntary contraction Strength in upper- and lower-body muscle groups. J Strength Cond Res [Internet]. 2014;28:3239–44 Available from: https://journals.lww. com/nsca-jscr/Fulltext/2014/11000/Effect_of_Caffeine_Ingestion_on_ Maximal_Voluntary.28.aspx.

9.Grgic J, Mikulic P. Caffeine ingestion acutely enhances muscular strength and power but not muscular endurance in resistance-trained men. Eur. J. Sport Sci. [Internet]. 2017 [cited 2018 Nov 21];17:1029–1036. Available from: https://www.tandfonline.com/doi/full/10.1080/17461391.2017.1330362.

10.Chia JS, Barrett LA, Chow JY, Burns SF. Effects of Caffeine Supplementation on Performance in Ball Games. Sport. Med. [Internet]. 2017 [cited 2019 Jan 5];47:2453–2471. Available from: http://link.springer.com/10.1007/s40279- 017-0763-6.

11.Astorino TA, Rohmann RL, Firth K. Effect of caffeine ingestion on one- repetition maximum muscular strength. Eur J Appl Physiol. 2008;102:127–32. 12.Brooks JH, Wyld  $\mathsf{K},$  Chrismas BCR. Acute effects of caffeine on Strength Performance in trained and untrained individuals. Enhanc: J. Athl; 2015.

13.Collier NB, Hardy MA, Millard-Stafford ML, Warren GL. Small beneficial effect of caffeinated energy drink ingestion on strength. J Strength Cond Res.

2016;30:1862–70.

14.Hahn CJ, Jagim AR, Camic CL, Andre MJ. The acute effects of a caffeine- containing supplement on anaerobic power and subjective measurements of fatigue in recreationally-active males. J. Strength Cond. Res. [Internet]. 2018;1. Available from: https://journals.lww.com/nsca-jscr/abstract/2006/ 08000/the_acute_effects_of_a_caffeine_containing.8.aspx.

15.Trevino M A, Coburn JW, Brown LE, Judelson D A, Malek MH. Acute effects of caffeine on Strength and muscle activation of the elbow flexors. J.

Strength Cond. Res. [Internet]. 2014;26:714–724. Available from: http://www. ncbi.nlm.nih.gov/pubmed/25029005.

16.Graham-Paulson TS, Perret C, Smith B, Crosland J, Goosey-Tolfrey VL. Nutritional Supplement Habits of Athletes with an Impairment and Their Sources of Information. Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2015 [cited 2018 Jun 10];25:387–95. Available from: http://journals.humankinetics.com/ doi/10.1123/ijsnem.2014-0155.

17.Arcoverde L, Silveira R, Tomazini F, Sansonio A, Bertuzzi R, Lima-Silva AE, et al. Effect of caffeine ingestion on anaerobic capacity quantified by different methods. Eynon N, editor. PLoS One [Internet]. 2017 [cited 2018 May 10];12: e0179457. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC5472322/pdf/pone.0179457.pdf.

18.Pataky MW, Womack CJ, Saunders MJ, Goffe JL, D’Lugos AC, El-Sohemy A, et al. Caffeine and 3-km cycling performance: Effects of mouth rinsing, genotype, and time of day. Scand. J. Med. Sci. Sports [Internet]. 2016 [cited 2019 Jan 15];26:613–619. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/26062916.



19.Rogers PJ, Hohoff C, Heatherley S V, Mullings EL, Maxfield PJ, Evershed RP, et al. Association of the Anxiogenic and Alerting Effects of Caffeine with ADORA2A and ADORA1 Polymorphisms and Habitual Level of Caffeine Consumption. Neuropsychopharmacology [Internet]. 2010 [cited 2019 Jan 15];35:1973–1983. Available from: http://www.ncbi.nlm.nih.gov/pubmed/ 20520601.

20.López-González LM, Sánchez-Oliver AJ, Mata F, Jodra P, Antonio J, Domínguez R. Acute caffeine supplementation in combat sports: a systematic review. J. Int. Soc. Sports Nutr. [Internet]. BioMed Central; 2018 [cited 2019 Jan 15];15:60. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/30594211.

21.Wager TD, Atlas LY. The neuroscience of placebo effects: Connecting context, learning and health. Nat. Rev. Neurosci. 2015;16:403–18.

22.Bérdi M, Köteles F, Hevesi K, Bárdos G, Szabo A. Elite athletes attitudes towards the use of placebo-induced performance enhancement in sports. Eur J Sport Sci. 2015;15:315–21.

23.Saunders B, de Oliveira LF, da Silva RP, de Salles Painelli V, Gonçalves LS, Yamaguchi G, et al. Placebo in sports nutrition: a proof-of-principle study involving caffeine supplementation. Scand. J. Med. Sci. Sports [Internet]. 2017 [cited 2018 May 10];27:1240–1247. Available from: http://doi.wiley. com/10.1111/sms.12793.

24.Beedie CJ, Stuart EM, Coleman DA, Foad AJ. Placebo Effects of Caffeine on Cycling Performance. Med. Sci. Sport. Exerc. [Internet]. 2006;38:2159–2164. Available from: https://insights.ovid.com/crossref?an=00005768-200612000- 00017.

25.Jefferies P, Gallagher P, Dunne S. The Paralympic athlete: a systematic review of the psychosocial literature. Prosthet. Orthot. Int. [Internet]. 2012 [cited 2018 may 17];36:278–289. Available from: http://www.ncbi.nlm.nih. gov/pubmed/22918904.

26.Tsarouhas K, Kioukia-Fougia N, Papalexis P, Tsatsakis A, Kouretas D, Bacopoulou F, et al. Use of nutritional supplements contaminated with banned doping substances by recreational adolescent athletes in Athens. Greece Food Chem Toxicol. 2018;115:447–50.

27.Van Thuyne W, Van Eenoo P, Delbeke FT. Nutritional supplements: prevalence of use and contamination with doping agents. Nutr. Res. Rev. [Internet]. 2006 [cited 2018 May 14];19:147. Available from: http://www. journals.cambridge.org/abstract_S0954422406000126.

28.Martínez-Sanz J, Sospedra I, Ortiz C, Baladía E, Gil-Izquierdo A, Ortiz- Moncada R. Intended or Unintended Doping? A Review of the Presence of Doping Substances in Dietary Supplements Used in Sports. Nutrients [Internet]. Multidisciplinary Digital Publishing Institute; 2017 [cited 2018 May 14];9:1093. Available from: http://www.mdpi.com/2072-6643/9/10/1093

29.Mathews NM. Prohibited contaminants in Dietary Supplements. Sports Health. 2018;10:19–30.

30.Geyer H, Parr MK, Koehler K, Mareck U, Schänzer W, Thevis M. Nutritional supplements cross-contaminated and faked with doping substances.

J. Mass Spectrom. 2008;43:892902.

31.Kraemer WJ, Fry AC. Strength testing: development and evaluation of methodology. In: Maud P, Nieman DC, editors. Fit. Sport. Med. A heal. Approach. 3rd ed. Palo Alto, CA: Bull Publishing; 1995. p. 11538.

32.Garnacho-Castaño MV, López-Lastra S, Maté-Muñoz JL. Reliability and validity assessment of a linear position transducer. J. Sport. Sci Med.

2014;14:128–36.

33.Cohen J. Statistical Power Analysis for the Behavioural Science (2nd Edition). In: Stat. Power Anaylsis Behav. Sci. 2nd ed; 1988. p. 25–7.

34.Baker D, Nance S, Moore M. The load that maximizes the average mechanical power output during explosive Bench Press throws in highly trained Athletes. J. Strength Cond. Res. 2001;15:204.

35.Hurst P, Foad A, Coleman D, Beedie C. Athletes Intending to Use Sports Supplements Are More Likely to Respond to a Placebo. Med. Sci. Sport. Exerc. [Internet]. 2017 [cited 2018 May 10];49:1877–1883. Available from: http://insights.ovid.com/crossref?an=00005768-201709000-00015.

36.Dascombe BJ, Karunaratna M, Cartoon J, Fergie B, Goodman C. Nutritional supplementation habits and perceptions of elite athletes within a state- based sporting institute. J. Sci. Med. Sport [Internet]. 2010 [cited 2018 may 20];13:274–280. Available from: http://www.ncbi.nlm.nih.gov/pubmed/ 19775936.

37.Tsitsimpikou C, Chrisostomou N, Papalexis P, Tsarouhas K, Tsatsakis A, Jamurtas A. The use of nutritional supplements among recreational athletes in Athens, Greece. Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2011 [cited 2018 may 20];21:377–384. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/21799216.



38.Darvishi L, Askari G, Hariri M, Bahreynian M, Ghiasvand R, Ehsani S, et al. The use of nutritional supplements among male collegiate athletes. Int. J. Prev. Med. [Internet]. 2013 [cited 2018 may 20];4:S68–S72. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/23717774.

39.Sousa M, Fernandes MJ, Moreira P, Teixeira VH. Nutritional Supplements Usage by Portuguese Athletes. Int. J. Vitam. Nutr. Res. [Internet]. 2013 [cited 2018 may 20];83:48–58. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/24220164.

40.Wardenaar FC, Ceelen IJM, Van Dijk J-W, Hangelbroek RWJ, Van Roy L, Van der Pouw B, et al. Nutritional Supplement Use by Dutch Elite and Sub-Elite Athletes: Does Receiving Dietary Counseling Make a Difference? Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2017 [cited 2018 may 20];27:32–42. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27615123.

41.Petróczi A, Naughton DP, Mazanov J, Holloway A, Bingham J. Performance enhancement with supplements: incongruence between rationale and practice. J. Int. Soc. Sports Nutr. [Internet]. 2007 [cited 2018 may 20];4:19. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17997853.

42.Petróczi A, Naughton DP, Mazanov J, Holloway A, Bingham J. Limited agreement exists between rationale and practice in athletes supplement use for maintenance of health: a retrospective study. Nutr. J. [Internet]. 2007 [cited 2018 may 20];6:34. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/17971239.

43.Duncan MJ, Stanley M, Parkhouse N, Cook K, Smith M. Acute caffeine ingestion enhances strength performance and reduces perceived exertion and muscle pain perception during resistance exercise. Eur. J. Sport Sci. [Internet]. 2013 [cited 2018 Nov 12];13:392–399. Available from: http://www. tandfonline.com/doi/abs/10.1080/17461391.2011.635811.

44.Tod DA, Iredale KF, McGuigan MR, Strange DEO, Gill N.Psyching-Up” Enhances Force Production During the Bench Press Exercise. J. Strength Cond. Res. [Internet]. 2005 [cited 2018 may 20];19:599. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/16095409.

45.Tod D, Iredale F, Gill N.Psyching-up” and muscular force production. Sports Med. [Internet]. 2003 [cited 2018 may 20];33:47–58. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/12477377.
# Effects of placebo on bench throw performance of Paralympic weightlifting athletes: a pilot study

Gustavo De Conti Teixeira Costa, Luan Galvão, Martim Bottaro, João Felipe Mota, Gustavo Duarte Pimentel and Paulo Gentil

## Abstract

Background: The aim of the present study was to analyse the effects of placebo on bench throw performance in Paralympic weightlifting athletes.

Methods: The study involved four Paralympic weightlifting male athletes (age:  $40.25\pm9.91$  years, weight:  $60.5\pm8.29\,\mathsf{k g},$  height:  $1.60\pm0.15\,\upmu\mathrm{m})$  that visited the laboratory in three occasions, separated by  $72\,\mathrm{h}$ . In the first session, the athletes were tested for bench press one repetition maximum (1RM). The other two sessions were performed in a randomized counter-balanced order and involved bench throw tests performed either after taking placebo while being informed that the capsule contained caffeine or without taking any substance (control). The bench throw tests were performed with loads corresponding to 50, 60, 70 and  $80\%$  of the bench press 1RM.

Results: According to the results, mean velocity  $(\Delta;0.08\,\mathsf{m}/\mathsf{s},\mathsf{E S}\,0.36,p<0.05)$ and mean propulsive velocity  at  $50\%$  of 1RM were significantly higher during placebo than control  $(p<0.05)$  $(\Delta;0.1\;1\;\mathsf{m}/\mathsf{s},\mathsf{E S}\;0.49,p<0.05)$ .

However, there were no difference between control and placebo for 60, 70 and  $80\%$  of 1RM  $(p>0.05)$ .

Conclusion: Our results suggest that placebo intake, when the athletes were informed they were taking caffeine, might be an efficient strategy to improve the performance of explosive movements in Paralympic weightlifting athletes when using low-loads. This brings the possibility of using placebo in order to increase performance, which might reduce the risks associated with ergogenic aids, such as side-effects and positive doping testing.

Keywords: Nutritional supplements, Sports performance, Psyching up

Background The intake of nutritional supplements are a common practiceamongphysicalactivitypractitionersand athletes; however, few actually have scientific evidence for their efficacy [1]. Among them, caffeine is one of the most consumed substances by athletes [2] and is consid- ered to positively impact physical performance [3]. In this regard, previous studies suggested that caffeine may be ergogenic, sparing muscle glycogen improving pain tolerance, reducing rate of perceived exertion, increase maximum voluntary contraction, strength and power in high-intensity activities besides of stimulating central nervous system [4–10]. However, there are specific controversies about its effects, mainly when the studies compared the acute effects of caffeine vs placebo intake showed inconsistent conclusions [11–17].



While genetic factors might explain a large portion of the variance associated with the caffeine effects such as pain tolerance, anxiogenic and alert effects [18–20], there are important psychological responses to ingesting a substance that should be considered. The placebo effect [21] can influence the physiological aspects to physical exercise performance [3], one of the all factors that might influence the effects of caffeine is the placebo effect [22]. In agreement with this, Saunders et al. [23] found an improvement in the cycling time to exhaustion in trained cyclists who ingested placebo believing to have ingested caffeine. Similarly, in a previous study, Beedie et al. [24] reported that when competitive male cyclists ingested placebo believing to have ingested caf- feine there were increases in aerobic power at VO2max test and  $10\mathrm{-km}$  time trials, with no difference in oxygen uptake, heart rate, and blood lactate. Such placebo ef- fects do not rule out a true effect of caffeine supplemen- tation. Although these studies suggested that aerobic performance improves following placebo intake when in- dividuals believed that they are ingesting caffeine, we are not aware of studies that measured the placebo effect in powerlifting. Moreover, Paralympic athletes have been shown to have psychological particularities that might make them especially vulnerable to the placebo effect, such as, concerns about having to perform consistently well throughout training and difficulties in coping with negative results [25]. Therefore, it seems important to perform specific studies in these group despite the diffi- culty to select subjects with these characteristics.



The analysis of the placebo effect might be of great practical importance since it could provide an alternative for improving performance through the intake of an inert substance, with no risk of testing positive for dop- ing or adverse effects, either due to the direct use of pro- hibited substances, or by the possible contamination of nutritional supplements [10, 26–30]. Based on this, the present study aimed to evaluate if muscle performance during explosive movements would change in Paralympic weightlifting athletes after the intake of placebo when the participants were informed that they were taking caffeine.

Materials and methods Participants FourParalympicweightliftingmaleathleteswere recruited to participate in this study (age:  $40.25\pm9.91~\mathrm{y},$  weight:  $60.5\pm8.29\,\mathrm{kg},$  height:  $1.60\pm0.15\,\mathrm{m})$ . One had dwarfism, one myelomeningocele and hydrocephalus and two poliomyelitis. Athletes trained regularly five times a week aiming to compete, and all had previous experience with caffeine use, but have not taken any caffeine supplements in the previous six months. Two athletes had won medals in at least two phases on the national circuit. They were only allowed to participate if they had no orthopaedic or cardiometabolic problems that could be aggravated by the study protocol, as attested by a physician. This study was approved by Federal University of Goias committee (2.058.322) and all the participants signed a written informed consent form before participation (Table 1).

Procedures The study is a randomized, double-blind, crossover study. The athletes attended the laboratory three times, with an interval of  $72\,\mathrm{h}$  between visits. During the first visit,theyweresubmittedtotheonerepetition maximum test (1RM) in the bench press, as previously recommended [31]. The second and third visits involved the bench throw tests. The athletes were randomly assigned to ingest a capsule of placebo or no capsule on the second and third visits, in a cross-over design. During the placebo situations, the athletes received one capsule containing maize starch one-hour prior the test and were informed that it contained  $6\,\mathrm{mg.kg-1}$  caffeine. The athletes were oriented to avoid caffeine containing beverages and foods one week prior to the beginning of the study.



Data collection Muscular performance was measured in the bench throw, using an isoinertial indicator (T-Force, Dynamic Measurement System; Ergotech Consulting S.L., Murcia, Spain). The exercise was performed on a smith machine and the athletes were instructed to perform three repeti- tions with maximum intended velocity in all repetitions. The tests were performed with 50, 60, 70 and  $80\%$  of 1RM, with  $5\,\mathrm{min}$  of rest between each load condition. The movement started with elbows fully extended and then the bar was get down until touching the sternum. A linear position transducer was attached to the bar. The bar position data were sampled at  $1000\,\mathrm{Hz}$  using a computer, as recommended by the manufacturer. The finite differentiation technique was used to calculate the velocity and acceleration of the bar, presenting an associated error of  $<0.25\%,$  while the displacement was accurate to  $\pm0.5\:\mathrm{mm}$  [32].

Statistical analysis Data were analysed using the Statistical Package of Social Science software (SPSS 20.0, Chicago, IL, USA). Factorial ANOVA with a within-within design was used to compare the performance between placebo vs control situation at different loads. When necessary multiple comparisons were used as post hoc. Data were consid- ered statistically significant when  $p<0.05$ . Effect size (ES) of the mean differences was determined using Cohen’s d. The magnitude of the ES was determined by Hopkin’s scale as follows:  $<0.1$  (trivial), 0.1–0.3 (small), 0.3–0.5 (moderate), 0.5–0.7 (large), 0.7–0.9 (very large) and  $>0.9$  (perfect) [33].

Results Table 2 shows the results relative to the absolute velocity values as a function of the load lifted. Although the mean velocity to peak and peak velocity were not different be- tween situations, the mean velocity at  $50\%$  RM was signifi- cantly higher in placebo vs control  $(\Delta\colon\,0.08\,\mathrm{m}/s)$ , with moderate effect size  $(0.36;~~p<0.05)$ . Similarly, mean propulsive velocity at  $50\%$  RM was significantly higher in placebo vs. control  $(\Delta\colon0.11\,\mathrm{m}/s)$  with moderate effect size  as shown in Fig. $(0.49;p<0.05)$  1.

Discussion To the best of our knowledge, this was the first study to analyse the placebo effects of caffeine on bench throw performance of Paralympic weightlifting athletes. Accord- ing to the results, the ingestion of placebo significantly increases mean velocity and mean propulsive velocity at  $50\%$  of 1RM. These findings may be particularly interest- ing, since this intensity is in the recommend range for maximumpoweroutputinthebenchthrow[34]. Moreover, considering that the tests involved highly trained athletes, the differences might be relevant to training and competition. It is important to note that, whilst they did not reach significance, the differences with higher loads oc- curred in the opposite direction, with a trend for a detri- mental effect with placebo. The reason for this is not known, but there are two hypotheses to consider. First, the tests were incremental, so it might be possible that the higher performance in the earlier sets with lower loads lead to fatigue in the later sets, performed with higher loads. Second, when the load increased and becomes more chal- lenging, the participants might have expected to have an improved performance perform. However, since there was not physiologic enhancement due to the supplementation, a negative psychological influence might have occurred.

Although the deception used in the present study is not common in scientific literature, it is closer to what happens in real world, where many athletes take nutri- tional supplements believing on a true physiological effect, which might affect the results [22]. Besides that, uptake placebo was able to improve performance on bench throw with no reports of adverse effects already shown to caffeine supplementation before [10]. In this regard, Hurst et al. suggested that the intention to im- prove performance by the athletes when taking placebo can make a difference in the final performance; there- fore, in order to take full advantage of this intervention, the athletes should believe in the benefits of the ingested substance [35].



Whilst many athletes believe that nutritional supple- ments are related to performance enhancements [36], most do not obtain adequate information and do not even know the active ingredients or mechanism of actionofthesubstancesused[36–42].Therefore, supplements use seems to rely more on beliefs than on scientific evidence. Considering our findings that an inertsubstancemightincreaseperformancewhen athletes were deceived to believe it was an ergogenic aid; this might help to explain the divergence that often occurs between anecdotal and scientific evidence. On the other hand, this study had the focus only in placebo effect and a third group of caffeine was not used aiming to compare three groups.



Regarding the possible explanation for the placebo effect, Beedie et al. divided it in four categories: pain reduction, belief-behaviour relation, attentional changes and arousal changes. Within these mechanisms, the improvements found in our study can be explained by attentional and arousal changes [24]. Besides that, caf- feine intake also increasing pain tolerance [43], however, this probably did not happen in this study due to short duration with just three repetitions of exercise per- formed. The placebo effect might be associated with self-directed cognitive strategies and preparatory arousal (i.e. including imagery and attentional focus), which has been shown to enhance force production [44, 45].

The major limitation of the present study is the low number of participants. However, due to the characteristics of the participants, it would be difficult to obtain a higher sample size. It would also be interesting to have a third situation, with caffeine use. Future research should be con- ducted on a higher number of athletes, including Paralym- pic athletes with different limitations. Moreover, it would be valuable to assess the long-term effects of placebo, in order to test if the regular increase in performance over training sessions might bring long-term benefits.



Conclusions Our results suggest that placebo intake, when the athletes believe they are taking caffeine, might be an efficient strategy to improve performance in the bench throw test in Paralympic weightlifting athletes under low-loads. This brings the possibility of using placebo in order to increase performance in plyometric and speed exercises, reducing the side effects and risks associated with the use of ergogenic aids. Additionally, it would be ideal that nutritional strategy was investigated with high loads before using it in practice. Finally, it might be suggested that part of the conflict that usually exists be- tween anecdotal reports and scientific evidence about nutritional supplementation can be associated to the psychological effects of ingesting a supplement.

Acknowledgments Authors would like to thank to subjects that participated of this study for their commitment and effort.

Funding This research received no grant.

Availability of data and materials The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Authors’ contributions GC participated in the study design, collected data, interpreted the data, and wrote the article. GC, LG, MB, JFM, GDP, and PG participated in the study design, and wrote the manuscript. All authors read and approved the final manuscript.

Ethics approval and consent to participate Participation in the study was voluntary, with written consent being obtained from each subject before the initiation of data collection. This study was conducted after review and approval by the Federal University of Goias committee. Committee’s reference number: 2.058.322.

Consent for publication All the study participants gave their consent to publish the research findings.

Competing interests The authors declare that they have no competing interests.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author details Faculdade de Educação Física e Dança, Universidade Federal de Goiás, Goiás, Brazil. Faculdade de Educação Física, Universidade de Brasília, Distrito Federal, Brasília, Brazil. Faculdade de Nutrição, Universidade Federal de Goiás, Goiás, Brazil.

### Received: 13 July 2018 Accepted: 12 February 2019

References 1.Maughan RJ, Burke LM, Dvorak J, Larson-Meyer DE, Peeling P, Phillips SM, et al. IOC consensus statement: dietary supplements and the high- performance athlete. Br. J. Sports Med. [Internet]. BMJ Publishing Group; 2018 [cited 2018 may 13];52:439–455. Available from: http://www.ncbi.nlm. nih.gov/pubmed/29540367.

2.Del Coso J, Muñoz G, Muñoz-Guerra J. Prevalence of caffeine use in elite athletes following its removal from the World Anti-Doping Agency list of banned substances. Appl. Physiol. Nutr. Metab. [Internet]. 2011;36:555–561. Available from: http://www.nrcresearchpress.com/doi/abs/10.1139/h11-052. 3.Lindheimer JB, O’Connor PJ, Dishman RK. Quantifying the Placebo Effect in Psychological Outcomes of Exercise Training: A Meta-Analysis of Randomized Trials. Sport. Med. 2015:693–711.

4.Lara B, Gonzalez-Millán C, Salinero JJ, Abian-Vicen J, Areces F, Barbero- Alvarez JC, et al. Caffeine-containing energy drink improves physical performance in female soccer players. Amino Acids. 2014;46:1385–92.

5.Fernandes-da-Silva J, Castagna C, Teixeira AS, Carminatti LJ, Guglielmo LGA. The peak velocity derived from the Carminatti test is related to physical match performance in young soccer players. J Sports Sci.

2016;34:2238–45.

6.Puente C, Abián-Vicén J, Salinero JJ, Lara B, Areces F, Del Coso J. Caffeine improves basketball performance in experienced basketball players.

Nutrients. 2017;9.

7.Goldstein E, Jacobs PL, Whitehurst M, Penhollow T, Antonio J. Caffeine enhances upper body strength in resistance-trained women. J Int Soc Sports Nutr. 2010;7.

8.Timmins TD, Saunders DH. Effect of caffeine ingestion on maximal voluntary contraction Strength in upper- and lower-body muscle groups. J Strength Cond Res [Internet]. 2014;28:3239–44 Available from: https://journals.lww. com/nsca-jscr/Fulltext/2014/11000/Effect_of_Caffeine_Ingestion_on_ Maximal_Voluntary.28.aspx.

9.Grgic J, Mikulic P. Caffeine ingestion acutely enhances muscular strength and power but not muscular endurance in resistance-trained men. Eur. J. Sport Sci. [Internet]. 2017 [cited 2018 Nov 21];17:1029–1036. Available from: https://www.tandfonline.com/doi/full/10.1080/17461391.2017.1330362.

10.Chia JS, Barrett LA, Chow JY, Burns SF. Effects of Caffeine Supplementation on Performance in Ball Games. Sport. Med. [Internet]. 2017 [cited 2019 Jan 5];47:2453–2471. Available from: http://link.springer.com/10.1007/s40279- 017-0763-6.

11.Astorino TA, Rohmann RL, Firth K. Effect of caffeine ingestion on one- repetition maximum muscular strength. Eur J Appl Physiol. 2008;102:127–32. 12.Brooks JH, Wyld  $\mathsf{K},$  Chrismas BCR. Acute effects of caffeine on Strength Performance in trained and untrained individuals. Enhanc: J. Athl; 2015.

13.Collier NB, Hardy MA, Millard-Stafford ML, Warren GL. Small beneficial effect of caffeinated energy drink ingestion on strength. J Strength Cond Res.

2016;30:1862–70.

14.Hahn CJ, Jagim AR, Camic CL, Andre MJ. The acute effects of a caffeine- containing supplement on anaerobic power and subjective measurements of fatigue in recreationally-active males. J. Strength Cond. Res. [Internet]. 2018;1. Available from: https://journals.lww.com/nsca-jscr/abstract/2006/ 08000/the_acute_effects_of_a_caffeine_containing.8.aspx.

15.Trevino M A, Coburn JW, Brown LE, Judelson D A, Malek MH. Acute effects of caffeine on Strength and muscle activation of the elbow flexors. J.

Strength Cond. Res. [Internet]. 2014;26:714–724. Available from: http://www. ncbi.nlm.nih.gov/pubmed/25029005.

16.Graham-Paulson TS, Perret C, Smith B, Crosland J, Goosey-Tolfrey VL. Nutritional Supplement Habits of Athletes with an Impairment and Their Sources of Information. Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2015 [cited 2018 Jun 10];25:387–95. Available from: http://journals.humankinetics.com/ doi/10.1123/ijsnem.2014-0155.

17.Arcoverde L, Silveira R, Tomazini F, Sansonio A, Bertuzzi R, Lima-Silva AE, et al. Effect of caffeine ingestion on anaerobic capacity quantified by different methods. Eynon N, editor. PLoS One [Internet]. 2017 [cited 2018 May 10];12: e0179457. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC5472322/pdf/pone.0179457.pdf.

18.Pataky MW, Womack CJ, Saunders MJ, Goffe JL, D’Lugos AC, El-Sohemy A, et al. Caffeine and 3-km cycling performance: Effects of mouth rinsing, genotype, and time of day. Scand. J. Med. Sci. Sports [Internet]. 2016 [cited 2019 Jan 15];26:613–619. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/26062916.



19.Rogers PJ, Hohoff C, Heatherley S V, Mullings EL, Maxfield PJ, Evershed RP, et al. Association of the Anxiogenic and Alerting Effects of Caffeine with ADORA2A and ADORA1 Polymorphisms and Habitual Level of Caffeine Consumption. Neuropsychopharmacology [Internet]. 2010 [cited 2019 Jan 15];35:1973–1983. Available from: http://www.ncbi.nlm.nih.gov/pubmed/ 20520601.

20.López-González LM, Sánchez-Oliver AJ, Mata F, Jodra P, Antonio J, Domínguez R. Acute caffeine supplementation in combat sports: a systematic review. J. Int. Soc. Sports Nutr. [Internet]. BioMed Central; 2018 [cited 2019 Jan 15];15:60. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/30594211.

21.Wager TD, Atlas LY. The neuroscience of placebo effects: Connecting context, learning and health. Nat. Rev. Neurosci. 2015;16:403–18.

22.Bérdi M, Köteles F, Hevesi K, Bárdos G, Szabo A. Elite athletes attitudes towards the use of placebo-induced performance enhancement in sports. Eur J Sport Sci. 2015;15:315–21.

23.Saunders B, de Oliveira LF, da Silva RP, de Salles Painelli V, Gonçalves LS, Yamaguchi G, et al. Placebo in sports nutrition: a proof-of-principle study involving caffeine supplementation. Scand. J. Med. Sci. Sports [Internet]. 2017 [cited 2018 May 10];27:1240–1247. Available from: http://doi.wiley. com/10.1111/sms.12793.

24.Beedie CJ, Stuart EM, Coleman DA, Foad AJ. Placebo Effects of Caffeine on Cycling Performance. Med. Sci. Sport. Exerc. [Internet]. 2006;38:2159–2164. Available from: https://insights.ovid.com/crossref?an=00005768-200612000- 00017.

25.Jefferies P, Gallagher P, Dunne S. The Paralympic athlete: a systematic review of the psychosocial literature. Prosthet. Orthot. Int. [Internet]. 2012 [cited 2018 may 17];36:278–289. Available from: http://www.ncbi.nlm.nih. gov/pubmed/22918904.

26.Tsarouhas K, Kioukia-Fougia N, Papalexis P, Tsatsakis A, Kouretas D, Bacopoulou F, et al. Use of nutritional supplements contaminated with banned doping substances by recreational adolescent athletes in Athens. Greece Food Chem Toxicol. 2018;115:447–50.

27.Van Thuyne W, Van Eenoo P, Delbeke FT. Nutritional supplements: prevalence of use and contamination with doping agents. Nutr. Res. Rev. [Internet]. 2006 [cited 2018 May 14];19:147. Available from: http://www. journals.cambridge.org/abstract_S0954422406000126.

28.Martínez-Sanz J, Sospedra I, Ortiz C, Baladía E, Gil-Izquierdo A, Ortiz- Moncada R. Intended or Unintended Doping? A Review of the Presence of Doping Substances in Dietary Supplements Used in Sports. Nutrients [Internet]. Multidisciplinary Digital Publishing Institute; 2017 [cited 2018 May 14];9:1093. Available from: http://www.mdpi.com/2072-6643/9/10/1093

29.Mathews NM. Prohibited contaminants in Dietary Supplements. Sports Health. 2018;10:19–30.

30.Geyer H, Parr MK, Koehler K, Mareck U, Schänzer W, Thevis M. Nutritional supplements cross-contaminated and faked with doping substances.

J. Mass Spectrom. 2008;43:892902.

31.Kraemer WJ, Fry AC. Strength testing: development and evaluation of methodology. In: Maud P, Nieman DC, editors. Fit. Sport. Med. A heal. Approach. 3rd ed. Palo Alto, CA: Bull Publishing; 1995. p. 11538.

32.Garnacho-Castaño MV, López-Lastra S, Maté-Muñoz JL. Reliability and validity assessment of a linear position transducer. J. Sport. Sci Med.

2014;14:128–36.

33.Cohen J. Statistical Power Analysis for the Behavioural Science (2nd Edition). In: Stat. Power Anaylsis Behav. Sci. 2nd ed; 1988. p. 25–7.

34.Baker D, Nance S, Moore M. The load that maximizes the average mechanical power output during explosive Bench Press throws in highly trained Athletes. J. Strength Cond. Res. 2001;15:204.

35.Hurst P, Foad A, Coleman D, Beedie C. Athletes Intending to Use Sports Supplements Are More Likely to Respond to a Placebo. Med. Sci. Sport. Exerc. [Internet]. 2017 [cited 2018 May 10];49:1877–1883. Available from: http://insights.ovid.com/crossref?an=00005768-201709000-00015.

36.Dascombe BJ, Karunaratna M, Cartoon J, Fergie B, Goodman C. Nutritional supplementation habits and perceptions of elite athletes within a state- based sporting institute. J. Sci. Med. Sport [Internet]. 2010 [cited 2018 may 20];13:274–280. Available from: http://www.ncbi.nlm.nih.gov/pubmed/ 19775936.

37.Tsitsimpikou C, Chrisostomou N, Papalexis P, Tsarouhas K, Tsatsakis A, Jamurtas A. The use of nutritional supplements among recreational athletes in Athens, Greece. Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2011 [cited 2018 may 20];21:377–384. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/21799216.



38.Darvishi L, Askari G, Hariri M, Bahreynian M, Ghiasvand R, Ehsani S, et al. The use of nutritional supplements among male collegiate athletes. Int. J. Prev. Med. [Internet]. 2013 [cited 2018 may 20];4:S68–S72. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/23717774.

39.Sousa M, Fernandes MJ, Moreira P, Teixeira VH. Nutritional Supplements Usage by Portuguese Athletes. Int. J. Vitam. Nutr. Res. [Internet]. 2013 [cited 2018 may 20];83:48–58. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/24220164.

40.Wardenaar FC, Ceelen IJM, Van Dijk J-W, Hangelbroek RWJ, Van Roy L, Van der Pouw B, et al. Nutritional Supplement Use by Dutch Elite and Sub-Elite Athletes: Does Receiving Dietary Counseling Make a Difference? Int. J. Sport Nutr. Exerc. Metab. [Internet]. 2017 [cited 2018 may 20];27:32–42. Available from: http://www.ncbi.nlm.nih.gov/pubmed/27615123.

41.Petróczi A, Naughton DP, Mazanov J, Holloway A, Bingham J. Performance enhancement with supplements: incongruence between rationale and practice. J. Int. Soc. Sports Nutr. [Internet]. 2007 [cited 2018 may 20];4:19. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17997853.

42.Petróczi A, Naughton DP, Mazanov J, Holloway A, Bingham J. Limited agreement exists between rationale and practice in athletes supplement use for maintenance of health: a retrospective study. Nutr. J. [Internet]. 2007 [cited 2018 may 20];6:34. Available from: http://www.ncbi.nlm.nih.gov/ pubmed/17971239.

43.Duncan MJ, Stanley M, Parkhouse N, Cook K, Smith M. Acute caffeine ingestion enhances strength performance and reduces perceived exertion and muscle pain perception during resistance exercise. Eur. J. Sport Sci. [Internet]. 2013 [cited 2018 Nov 12];13:392–399. Available from: http://www. tandfonline.com/doi/abs/10.1080/17461391.2011.635811.

44.Tod DA, Iredale KF, McGuigan MR, Strange DEO, Gill N.Psyching-Up” Enhances Force Production During the Bench Press Exercise. J. Strength Cond. Res. [Internet]. 2005 [cited 2018 may 20];19:599. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/16095409.

45.Tod D, Iredale F, Gill N.Psyching-up” and muscular force production. Sports Med. [Internet]. 2003 [cited 2018 may 20];33:47–58. Available from: http:// www.ncbi.nlm.nih.gov/pubmed/12477377.
### ISSN: 1941-5982 (Print) 1941-5990 (Online) Journal homepage: http://www.tandfonline.com/loi/tphm15

# On some specimens of pseudomorphs in the imperial museum of vienna

## Dr. Tschermak

To cite this article: Dr. Tschermak (1863) On some specimens of pseudomorphs in the imperial museum of vienna, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 25:168, 323-323, DOI: 10.1080/14786446308643467

## To link to this article:  https://doi.org/10.1080/14786446308643467

### Intelligence and Miscellaneous Articles. 323 

on the other, and the vapour acts in a similar manner, so that only  one side at the same time is affected by each.  Your obedient servant,  ThoMas Woo\~s.  Parsonstown, February 1863. 

ON SOME SPECIMENS OF PSEUDOMORPHS IN THE IMPERIAL 

MUSEUM OF VIENNA\*. BY DR. TSCHERMAK.  The Imperial Museum of Vienna contains many very curious  specimens of Pseudomorphs, and among them are several calculated  to throw light on controverted geological questions. In a hand-  specimen of gneiss from the Rathhaus-Berg, near Gastein, amphi-  bole has replaced mica. This gneiss contains altered amphibole in  the interior of masses of mica; and it may therefore be inferred,  with a certain degree of probability, that the whole of this gneiss,  whose felspathic components likewise bear traces of metamorphism,  is an altered amphibolic rock. The same, so far as may be inferred  from cabinet-specimens, may be the case with other varieties of  gneiss, especially with those of Brazil. 

Pseudomorphs of compact felspar, replacing crystals of the same  mineral species, occur in the antique green porphyry of Italy and  Egypt, as also in the diabasic porphyry from the Hartz,--a fact  not unimportant in regard to the origin of certain varieties of this  group of rocks. A double Pseudomorph--crystallized gypsum meta-  morphosed into fibrous gypsum, and this, in the course of time, into  fibrous quartz--occurrlng, together with spaces left by the decom-  position of crystallized iron pyrites, in a specimen of chloritie argil-  laceous slate from the Eifel, may suggest some of the processes  undergone by this rock before it arrived at its present condition.  Some other cases of Pseudomorphs, as specular oxide of iron re-  placing olivine in the trappean rocks around Edinburgh; opal re-  placing nephrite, from Elbingerode ; opal replacing augite, from the  same locality ; and calcareous spar having taken the place of augite  in the metamorphosed pyroxenic porphyry of T6kSr5 (Transylvania),  have been neither noticed nor described before.--Proc. Imp. Acad.  Vienna, 
### ISSN: 1941-5982 (Print) 1941-5990 (Online) Journal homepage: http://www.tandfonline.com/loi/tphm15

# On some specimens of pseudomorphs in the imperial museum of vienna

## Dr. Tschermak

To cite this article: Dr. Tschermak (1863) On some specimens of pseudomorphs in the imperial museum of vienna, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 25:168, 323-323, DOI: 10.1080/14786446308643467

## To link to this article:  https://doi.org/10.1080/14786446308643467

### Intelligence and Miscellaneous Articles. 323 

on the other, and the vapour acts in a similar manner, so that only  one side at the same time is affected by each.  Your obedient servant,  ThoMas Woo\~s.  Parsonstown, February 1863. 

ON SOME SPECIMENS OF PSEUDOMORPHS IN THE IMPERIAL 

MUSEUM OF VIENNA\*. BY DR. TSCHERMAK.  The Imperial Museum of Vienna contains many very curious  specimens of Pseudomorphs, and among them are several calculated  to throw light on controverted geological questions. In a hand-  specimen of gneiss from the Rathhaus-Berg, near Gastein, amphi-  bole has replaced mica. This gneiss contains altered amphibole in  the interior of masses of mica; and it may therefore be inferred,  with a certain degree of probability, that the whole of this gneiss,  whose felspathic components likewise bear traces of metamorphism,  is an altered amphibolic rock. The same, so far as may be inferred  from cabinet-specimens, may be the case with other varieties of  gneiss, especially with those of Brazil. 

Pseudomorphs of compact felspar, replacing crystals of the same  mineral species, occur in the antique green porphyry of Italy and  Egypt, as also in the diabasic porphyry from the Hartz,--a fact  not unimportant in regard to the origin of certain varieties of this  group of rocks. A double Pseudomorph--crystallized gypsum meta-  morphosed into fibrous gypsum, and this, in the course of time, into  fibrous quartz--occurrlng, together with spaces left by the decom-  position of crystallized iron pyrites, in a specimen of chloritie argil-  laceous slate from the Eifel, may suggest some of the processes  undergone by this rock before it arrived at its present condition.  Some other cases of Pseudomorphs, as specular oxide of iron re-  placing olivine in the trappean rocks around Edinburgh; opal re-  placing nephrite, from Elbingerode ; opal replacing augite, from the  same locality ; and calcareous spar having taken the place of augite  in the metamorphosed pyroxenic porphyry of T6kSr5 (Transylvania),  have been neither noticed nor described before.--Proc. Imp. Acad.  Vienna, 
### ISSN: 1941-5982 (Print) 1941-5990 (Online) Journal homepage: http://www.tandfonline.com/loi/tphm15

# On some specimens of pseudomorphs in the imperial museum of vienna

## Dr. Tschermak

To cite this article: Dr. Tschermak (1863) On some specimens of pseudomorphs in the imperial museum of vienna, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 25:168, 323-323, DOI: 10.1080/14786446308643467

## To link to this article:  https://doi.org/10.1080/14786446308643467

### Intelligence and Miscellaneous Articles. 323 

on the other, and the vapour acts in a similar manner, so that only  one side at the same time is affected by each.  Your obedient servant,  ThoMas Woo\~s.  Parsonstown, February 1863. 

ON SOME SPECIMENS OF PSEUDOMORPHS IN THE IMPERIAL 

MUSEUM OF VIENNA\*. BY DR. TSCHERMAK.  The Imperial Museum of Vienna contains many very curious  specimens of Pseudomorphs, and among them are several calculated  to throw light on controverted geological questions. In a hand-  specimen of gneiss from the Rathhaus-Berg, near Gastein, amphi-  bole has replaced mica. This gneiss contains altered amphibole in  the interior of masses of mica; and it may therefore be inferred,  with a certain degree of probability, that the whole of this gneiss,  whose felspathic components likewise bear traces of metamorphism,  is an altered amphibolic rock. The same, so far as may be inferred  from cabinet-specimens, may be the case with other varieties of  gneiss, especially with those of Brazil. 

Pseudomorphs of compact felspar, replacing crystals of the same  mineral species, occur in the antique green porphyry of Italy and  Egypt, as also in the diabasic porphyry from the Hartz,--a fact  not unimportant in regard to the origin of certain varieties of this  group of rocks. A double Pseudomorph--crystallized gypsum meta-  morphosed into fibrous gypsum, and this, in the course of time, into  fibrous quartz--occurrlng, together with spaces left by the decom-  position of crystallized iron pyrites, in a specimen of chloritie argil-  laceous slate from the Eifel, may suggest some of the processes  undergone by this rock before it arrived at its present condition.  Some other cases of Pseudomorphs, as specular oxide of iron re-  placing olivine in the trappean rocks around Edinburgh; opal re-  placing nephrite, from Elbingerode ; opal replacing augite, from the  same locality ; and calcareous spar having taken the place of augite  in the metamorphosed pyroxenic porphyry of T6kSr5 (Transylvania),  have been neither noticed nor described before.--Proc. Imp. Acad.  Vienna, 
### ISSN: 1941-5982 (Print) 1941-5990 (Online) Journal homepage: http://www.tandfonline.com/loi/tphm15

# On some specimens of pseudomorphs in the imperial museum of vienna

## Dr. Tschermak

To cite this article: Dr. Tschermak (1863) On some specimens of pseudomorphs in the imperial museum of vienna, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 25:168, 323-323, DOI: 10.1080/14786446308643467

## To link to this article:  https://doi.org/10.1080/14786446308643467

### Intelligence and Miscellaneous Articles. 323 

on the other, and the vapour acts in a similar manner, so that only  one side at the same time is affected by each.  Your obedient servant,  ThoMas Woo\~s.  Parsonstown, February 1863. 

ON SOME SPECIMENS OF PSEUDOMORPHS IN THE IMPERIAL 

MUSEUM OF VIENNA\*. BY DR. TSCHERMAK.  The Imperial Museum of Vienna contains many very curious  specimens of Pseudomorphs, and among them are several calculated  to throw light on controverted geological questions. In a hand-  specimen of gneiss from the Rathhaus-Berg, near Gastein, amphi-  bole has replaced mica. This gneiss contains altered amphibole in  the interior of masses of mica; and it may therefore be inferred,  with a certain degree of probability, that the whole of this gneiss,  whose felspathic components likewise bear traces of metamorphism,  is an altered amphibolic rock. The same, so far as may be inferred  from cabinet-specimens, may be the case with other varieties of  gneiss, especially with those of Brazil. 

Pseudomorphs of compact felspar, replacing crystals of the same  mineral species, occur in the antique green porphyry of Italy and  Egypt, as also in the diabasic porphyry from the Hartz,--a fact  not unimportant in regard to the origin of certain varieties of this  group of rocks. A double Pseudomorph--crystallized gypsum meta-  morphosed into fibrous gypsum, and this, in the course of time, into  fibrous quartz--occurrlng, together with spaces left by the decom-  position of crystallized iron pyrites, in a specimen of chloritie argil-  laceous slate from the Eifel, may suggest some of the processes  undergone by this rock before it arrived at its present condition.  Some other cases of Pseudomorphs, as specular oxide of iron re-  placing olivine in the trappean rocks around Edinburgh; opal re-  placing nephrite, from Elbingerode ; opal replacing augite, from the  same locality ; and calcareous spar having taken the place of augite  in the metamorphosed pyroxenic porphyry of T6kSr5 (Transylvania),  have been neither noticed nor described before.--Proc. Imp. Acad.  Vienna, 
# Planar Hall effect in Hall sensors made from InP/InGaAs heterostructure

## M. Morvic∗, J. Betko

Institute of Electrical Engineering, Slovak Academy of Sciences, 841 03 Bratislava, Slovakia Received 13 June 2003; received in revised form 24 November 2004; accepted 24 November 2004 Available online 6 January 2005



### Abstract

Room temperature planar Hall effect is investgated in a Hall sensor prepared from InP/InGaAs heterostructure containing triangel quantum well with two dimensional electron gas (2DEG) and is compared with that in a bulk GaAs Hall sensor. The heterostructure sensor investigated exhibits nearly three order of magnitude smaller planar Hall coefﬁcient than the value corresponding to the transversal physical magnetore- sistance coefﬁcient. This is associated with a low planar magnetoresistance effect due to an extremely low thicknes of the electrically active layer. The ratio of the planar- to the transversal Hall voltage in the heterostructure Hall sensor is found to be lower by a factor of 50 than that in the bulk one. Thus, Hall sensors made from heterostructures with 2DEG are particularly suitable as elements of systems for precise measurements of arbitrarily oriented magnetic ﬁeld.

 $\copyright$  2004 Elsevier B.V. All rights reserved.

Keywords: Magnetic ﬁeld measurement; Planar Hall effect; Magnetoresistance; Heterostructure Hall sensor

### 1. Introduction

The planar Hall effect [1–4] is an important issue con- cerning measurements of the magnetic ﬁeld vector using Hall sensors. It is a disturbing effect in this case and contributes to the measurement errors. This is a reason to search for new arrangements of Hall sensors in which the planar Hall ef- fect is depressed. Schott et al. [2] presented recently a new type vertical Hall sensor in which the experimental value of the planar Hall effect is about 10 times smaller than theory predicts.

In this paper, we present room temperature (about  $296\,\mathrm{K}$ ) experimental data of planar Hall effect and magnetoresis- tance measured in a Hall sensor prepared from InP/InGaAs heterostructure containing quantum well with two dimen- sional electron gas (2DEG). The results are compared with those obtained in a bulk GaAs Hall sensor.

### 2. Planar Hall effect in Hall sensors

We assume GaAs or InGaAs rectangle Hall sensor made from  $(0\,0\,1)$  oriented wafers sliced in [1 1 0] and related di- rections. We also assume isotropic behaviour of the materials studied, without anisotropic effects as those shown by Burger [5]. The voltage  $U_{34}$  between the electrodes 3 and 4 for a rectangular Hall device (Fig. 1) in low electric and magnetic ﬁelds can be written as [2,3]: sheet Hall coefﬁcients, respectively;  $I$ , the current ﬂowing from electrode 1 to electrode 2 (i.e. in-direction); $x$   $B_{x}\!=\!B$   and $\sin\theta\,\cos\phi,\,B_{y}\,{=}\,B\,\,\sin\theta\,\,\sin\phi$   $B_{z}\!=\!B\,\cos\theta$  are the  $x,\,y$ , and components of the magnetic ﬁeld, respectively; $z$  is the $\theta$  angle between the magnetic ﬁeld vector and the $B$ -axis and $z$   is the angle between the planar component of the magnetic $\phi$  ﬁeld and the $B_{x y}$ -axis. $x$  Measurements at both positive and negative orientations of the magnetic ﬁeld  $(B_{+}$  and  $B_{-}$ ) allow us to separate the planar Hall voltage as follows:




$$
U_{\mathrm{H,plan}}=\frac{U_{34}(B_{+})+U_{34}(B_{-})}{2}-U_{\mathrm{asym}}
$$


(4) Applying a low magnetic ﬁeld  $(\mu_{\mathrm{H}}\,B\leq1;\,\mu_{\mathrm{H}}$  is the Hall mobility)  $U_{\mathrm{asym}}$  can be supposed to be a constant and can be obtained from the measurement at  $B\!=\!0$  (in general,  $U_{\mathrm{asym}}$  increases with increasing magnetic ﬁeld due to the magne- toresistance effect). The amplitude of the planar Hall voltage  can be obtained experimentally by turning the $U_{\mathrm{H,plan,max}}$  sensor around the  $z$ -axis under a magnetic ﬁeld applied in the xy plane  $(\theta\!\cong\!90^{\circ}$ ; further denoted as planar case). In the planar case, the dependence of  $U_{\mathrm{H,plan}}$  on the angle can be $\phi$  ﬁtted by the following ﬁtting function: (5)

 



### 
$$
U_{\mathrm{H,plan}}=P_{1}\mathrm{sin}2\phi
$$


[6], given as

### 
$$
C=\frac{\left\langle\tau^{3}\right\rangle\left\langle\tau\right\rangle}{\left\langle\tau^{2}\right\rangle^{2}},
$$


(8) where  $\tau$  is the relaxation time.  $C$  can be obtained experimen- tally at (transversal case) as $\theta\!=\!0$  [6]:



### 
$$
C\cong\frac{(\Delta\rho/\rho_{0})_{\mathrm{transv}}}{(\mu_{\mathrm{H}}B)^{2}}
$$


(9)

Therefore, the planar sheet Hall coefﬁcient can be ob- tained directly from the Hall measurement (Eq. (6)) and it can be compared with the value calculated using the physical magnetoresistance coefﬁcient (Eqs. (7) and (9).

### 3. Experimental

Two samples were used to compare the planar Hall effect in Hall sensors prepared from a heterostructure with 2DEG (sample 1) and from bulk semiconductor (sample 2). For sam- ple 1 heterostructure InP/InGaAs prepared by MOCVD was used; a six arms Hall bar (Fig. 1), cca 5 mm long and $0.4\,\mathrm{mm}$  wide, was shaped by photolithography. The presence of the 2DEG in the InP/InGaAs heterostructure used for sample 1 was conﬁrmed previously by measurements of the quan- tum Hall effect at $4.2\,\mathrm{K}$  in high magnetic ﬁeld [7]. Sample 2 was made from a bulk GaAs wafer; the Hall bar was $0.5\,\mathrm{mm}$  thick,  $14\,\mathrm{mm}$  long and $2.5\,\mathrm{mm}$  wide with a contact arrange- ment like that in Fig. 1 (two contacts for the supplied current and four point contacts for the Hall effect and the resistivity measurements). In fact, we measured two more samples with thickness 7 and  $15\,\upmu\mathrm{m}$ . For the sake of clarity we show only the two ones with extreme thickness.

The basic room-temperature transport parameters of the samples are shown in Table 1;  $\mu_{\mathrm{H}}$  is the Hall mobility,  $\rho_{\mathrm{s}}$  is the sheet resistivity,  $n_{\mathrm{H,s}}$  is the sheet Hall concentration and  $U_{\mathrm{H,transv}}/(B I)$  is the sensitivity of the Hall sensor at $\theta\!=\!0$  (transversal case).

### 4. Results and discussion

Voltages  $U_{34}$  and  $U_{56}$  were measured in the dark to obtain the planar Hall coefﬁcient and the magnetoresistance param- eters, respectively. Magnetic ﬁeld of both polarities up to 1 T was applied in various directions with regard to the sen- sor plane. Supply currents of  $0.1\,\mathrm{mA}$  and  $10\,\mathrm{mA}$  were used for the heterostructure- and the bulk Hall sensors, respec- tively.

In Fig. 2, the planar Hall voltage versus the angle  $\phi$  between  $B$  and  $I$  for the 2DEG InP/InGaAs Hall sensor is plotted in the planar case  $(\theta\!=\!90^{\circ})$ ) at  $B\,{=}\,0.7$ , 0.8, 0.9 and  $1.0\,\mathrm{T}.$ . The ﬁtting curves (see Eq. (5)) for the individual magnetic ﬁelds are also shown. From the ﬁtting procedure values 2.6, 3.4, 4.3 and  $5.4\,\upmu\mathrm{V}$  were obtained for  $U_{\mathrm{H,plan,max}}$  at  $B\!=\!0.7$ , 0.8, 0.9 and  $1.0\,\mathrm{T},$  respectively. The dependence of  $U_{\mathrm{H,plan,max}}$  versus yields $B$    $U_{\mathrm{H,plan,max}}/(B^{2}I)\!=\!0.053\,\mathrm{V}\,\mathrm{T}^{-2}\,\mathrm{A}^{-1}$ which corresponds to a planar sheet Hall coefﬁcient  $R_{\mathrm{H,plan,}s}\,{=}\,0.11\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2}$  (see Eq. (6)). We observed the typical curve for the planar Hall voltage, with extrema at  $45^{\circ},\ 135^{\circ},\ 225^{\circ}$  and  $315^{\circ}$ , as it is given by Eq. (5) and shown in Fig. 2, without any anisotropy, as it was observed in Si Hall sensors [5].



InFig.3,theexperimentalresultsofthemagnetoresistance versus the angle  $\theta$  are plotted for the both samples and are ﬁtted by the following curve (10) where and $A_{1}$  are ﬁtting parameters representing the phys- $A_{2}$  ical magnetoresistance coefﬁcient  $C$  and the planar magne- toresistanceparameter $(\Delta\rho/\rho_{0})_{\mathrm{plan}}/(\mu_{\mathrm{H}}\,B)^{2}$ ,respectively.For the 2DEG InP/InGaAs sensor values 0.053 and 0.000015 were obtained for the physical magnetoresistance coefﬁcient and the planar magnetoresistance parameter, respectively. In the bulk GaAs sensor, the dependence of the magnetore- sistance parameter on the angle  $\theta$  is nearly constant and the magnetoresistance does not express signiﬁcant differ- ence between transversal and planar positions  $(A_{1}\!=\!0.018$ , ). While in the sensor from bulk semiconductor $A_{2}\,{=}\,0.015)$  we obtained $A_{1}\!\cong\!A_{2}\!\cong\!C$  $\mathrm{A}1\,{\overset{\sim}{=}}\,\mathbf{C}$ erostructure sensor exhibi $\theta$  large difference between A1 ∼= and  $\phi$  adjusted at the measurements are not exactly identi- cal with that deﬁned above. A small deviation of the starting values  $\theta_{0}$  or/and  $\phi_{0}$  from zero was accepted in the ﬁtting procedure.

 



The results obtained for InP/InGaAs heterostructure Hall sensor as well as for the bulk GaAs Hall sensor are sum- marized in Table 2. The column  $\rho_{\mathrm{s}}\ \ \mu_{\mathrm{H}}{}^{2}\,C$  represents a theoretical value for the planar Hall coefﬁcient calculated from Eq. (7) using the physical magnetoresistance coefﬁ- cient. As it can be seen, the planar sheet Hall coefﬁcient in the InP/InGaAs heterostructure sensor  $(0.11\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  is nearly three order of magnitude lower than that cor- responding to the physical magnetoresistance coefﬁcient . This is associated with a low magnetore- $(76\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  sistance in the planar case due to the extremely low thick- ness of the electrically active layer. The planar Hall sheet coefﬁcient in the bulk GaAs sensor  $(0.00\bar{36}\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  is comparable with that corresponding to the physical magne- toresistance coefﬁcient  $(0.0058\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$ . The ratio of  and $U_{\mathrm{H,plan,max}}$   $U_{\mathrm{H,transv}}$  proportional to  $B$  with a constant of the proportionality  $4.2\times10^{-5}\,\mathrm{T}^{-1}$  and  $2.5\times10^{-3}\,\mathrm{T}^{-1}$  was obtained for the heterostructure- and bulk sensors, re- spectively. The unambiguos growing difference between the measured planar Hall coefﬁcient and that determined from thephysicalmagnetoresistencewithdecreasingsamplethick- ness was observed also in the samples not shown here. So, the lower the sensor thickness the greater the differ- ence between the planar Hall constants values determined from planar Hall voltage and the magnetoresistance mea- surements.



### 5. Summary

Room temperature planar Hall effect and magnetoresis- tance were measured and analysed in Hall sensors prepared from an InP/InGaAs heterostructure as well as from a bulk GaAs. Both polarities of the magnetic ﬁeld were utilized at each position of the magnetic ﬁeld adjusted with regard to the sample plane, including the transversal- and the planar positions, to obtain correct values for the planar sheet Hall coefﬁcient and the physical magnetoresistance coefﬁcient, respectively.

In the heterostructure Hall sensor, the planar sheet Hall coefﬁcient was found to be about three orders of magnitude lower than that corresponding to the theoretical value calcu- lated using the transversal magnetoresistance parameter. This is associated with extremely low thickness of the electrically active layer in these sensors. The ratio of the maximum planar Hall voltage and the maximum transversal Hall voltage was about 50 times lower in the heterostructure sensor than that in the bulk Hall sensor at the same. From this point of view $B$  Hall sensors prepared from heterostructures with 2DEG are particularly suitable to be used in systems for precise mea- surement of magnetic ﬁeld vectors.

### Acknowledgments

Authors thank S. Hasen¨ohrl for the preparing of the InP/InGaAs heterostructure, D. Greguˇsov´a for the litogra- phy shaping and M Polak for helpful discussions. This work has been supported by Grant No. 2/3115/23 from the Slovak Grant Agency VEGA.

### References

[1] C. Goldberg, R.E. Davis, New galvanomagnetic effect, Phys. Rev. 94 (1954) 1121–1125. [2] Ch. Schott, P.-A. Besse, R.S. Popovic, Planar Hall effect in the vertical Hall sensor, Sens. Actuators 85 (2000) 111–115. [3] M. Polak, I. Hlasnik, Planar Hall effect in heavy doped  $n$ -InSb and its inﬂuence on the measurement of magnetic ﬁeld components with Hall generator at  $4.2\,\mathrm{K}$ , Solid-State Electron. 13 (1970) 219– 227.



[4] K. Merinsky, Ph.D. Thesis, Institute of Electrical Engineering of the Slovak Academy of Sciences, Bratislava, 1962.

[5] F. Burger, High precision miniaturized magnetic angular encoder, Ph.D. Thesis, (EPFL No. 2269) Hartung-Gore Konstanz, Germany, ISBN: 3-89649-629-8, 2001 Series in Microsystems Vol. 11.

[6] D.C. Look, Electrical characterization of GaAs materials and devices, Wiley, Chicester, 1989, p. 61.

[7] M. Morvic, J. Betko, S. Hasen¨ohrl, D. Greguˇsov´a, V. Cambel, P. Eli´aˇs, J. Nov´ak, On quantum Hall resistance and Shubnikov de Haas effect measurements on InP/InGaAs structures, in: P. Kordoˇs, J. Nov´ak (Eds.), Proceedings of the 3rd International Workshop on Heterostruc- ture Epitaxy and Devices-HEAD’97, NATO Sciences Series 3, vol. 48, Kluwer Academic Publishers, Dordrecht, 1998, p. 285.

### Biographies

Marian Morvic and his Ph.D. from the Technical University of Bratislava (Slovakia) in 1963 and 1973, respectively. He has worked since 1962 up to now as a research and scientiﬁc worker at the Institute of Electrical Engineering of the Slovak Academy of Sciences. He worked on the preparation of III-V semiconducting materials and devices as well as on their characterization. His current research interest includes characterization of basic electrical transport parameters of semiconductor epitaxial layers.

J´ulius Betko 1960 and the CSc. (equivalent to Ph.D.) degree in Electronics in 1971, both from the Technical University, Bratislava (Slovakia). In 1992, he received the Dr.Sc. degree from the Slovak Academy of Sciences. Since 1960 he has been with Institute of Electrical Engineering of the Slovak Academy of Sciences where is working on electrical characterization of semiconductors and semiconductor structures.
# Planar Hall effect in Hall sensors made from InP/InGaAs heterostructure

## M. Morvic∗, J. Betko

Institute of Electrical Engineering, Slovak Academy of Sciences, 841 03 Bratislava, Slovakia Received 13 June 2003; received in revised form 24 November 2004; accepted 24 November 2004 Available online 6 January 2005



### Abstract

Room temperature planar Hall effect is investgated in a Hall sensor prepared from InP/InGaAs heterostructure containing triangel quantum well with two dimensional electron gas (2DEG) and is compared with that in a bulk GaAs Hall sensor. The heterostructure sensor investigated exhibits nearly three order of magnitude smaller planar Hall coefﬁcient than the value corresponding to the transversal physical magnetore- sistance coefﬁcient. This is associated with a low planar magnetoresistance effect due to an extremely low thicknes of the electrically active layer. The ratio of the planar- to the transversal Hall voltage in the heterostructure Hall sensor is found to be lower by a factor of 50 than that in the bulk one. Thus, Hall sensors made from heterostructures with 2DEG are particularly suitable as elements of systems for precise measurements of arbitrarily oriented magnetic ﬁeld.

 $\copyright$  2004 Elsevier B.V. All rights reserved.

Keywords: Magnetic ﬁeld measurement; Planar Hall effect; Magnetoresistance; Heterostructure Hall sensor

### 1. Introduction

The planar Hall effect [1–4] is an important issue con- cerning measurements of the magnetic ﬁeld vector using Hall sensors. It is a disturbing effect in this case and contributes to the measurement errors. This is a reason to search for new arrangements of Hall sensors in which the planar Hall ef- fect is depressed. Schott et al. [2] presented recently a new type vertical Hall sensor in which the experimental value of the planar Hall effect is about 10 times smaller than theory predicts.

In this paper, we present room temperature (about  $296\,\mathrm{K}$ ) experimental data of planar Hall effect and magnetoresis- tance measured in a Hall sensor prepared from InP/InGaAs heterostructure containing quantum well with two dimen- sional electron gas (2DEG). The results are compared with those obtained in a bulk GaAs Hall sensor.

### 2. Planar Hall effect in Hall sensors

We assume GaAs or InGaAs rectangle Hall sensor made from  $(0\,0\,1)$  oriented wafers sliced in [1 1 0] and related di- rections. We also assume isotropic behaviour of the materials studied, without anisotropic effects as those shown by Burger [5]. The voltage  $U_{34}$  between the electrodes 3 and 4 for a rectangular Hall device (Fig. 1) in low electric and magnetic ﬁelds can be written as [2,3]: sheet Hall coefﬁcients, respectively;  $I$ , the current ﬂowing from electrode 1 to electrode 2 (i.e. in-direction); $x$   $B_{x}\!=\!B$   and $\sin\theta\,\cos\phi,\,B_{y}\,{=}\,B\,\,\sin\theta\,\,\sin\phi$   $B_{z}\!=\!B\,\cos\theta$  are the  $x,\,y$ , and components of the magnetic ﬁeld, respectively; $z$  is the $\theta$  angle between the magnetic ﬁeld vector and the $B$ -axis and $z$   is the angle between the planar component of the magnetic $\phi$  ﬁeld and the $B_{x y}$ -axis. $x$  Measurements at both positive and negative orientations of the magnetic ﬁeld  $(B_{+}$  and  $B_{-}$ ) allow us to separate the planar Hall voltage as follows:




$$
U_{\mathrm{H,plan}}=\frac{U_{34}(B_{+})+U_{34}(B_{-})}{2}-U_{\mathrm{asym}}
$$


(4) Applying a low magnetic ﬁeld  $(\mu_{\mathrm{H}}\,B\leq1;\,\mu_{\mathrm{H}}$  is the Hall mobility)  $U_{\mathrm{asym}}$  can be supposed to be a constant and can be obtained from the measurement at  $B\!=\!0$  (in general,  $U_{\mathrm{asym}}$  increases with increasing magnetic ﬁeld due to the magne- toresistance effect). The amplitude of the planar Hall voltage  can be obtained experimentally by turning the $U_{\mathrm{H,plan,max}}$  sensor around the  $z$ -axis under a magnetic ﬁeld applied in the xy plane  $(\theta\!\cong\!90^{\circ}$ ; further denoted as planar case). In the planar case, the dependence of  $U_{\mathrm{H,plan}}$  on the angle can be $\phi$  ﬁtted by the following ﬁtting function: (5)

 



### 
$$
U_{\mathrm{H,plan}}=P_{1}\mathrm{sin}2\phi
$$


[6], given as

### 
$$
C=\frac{\left\langle\tau^{3}\right\rangle\left\langle\tau\right\rangle}{\left\langle\tau^{2}\right\rangle^{2}},
$$


(8) where  $\tau$  is the relaxation time.  $C$  can be obtained experimen- tally at (transversal case) as $\theta\!=\!0$  [6]:



### 
$$
C\cong\frac{(\Delta\rho/\rho_{0})_{\mathrm{transv}}}{(\mu_{\mathrm{H}}B)^{2}}
$$


(9)

Therefore, the planar sheet Hall coefﬁcient can be ob- tained directly from the Hall measurement (Eq. (6)) and it can be compared with the value calculated using the physical magnetoresistance coefﬁcient (Eqs. (7) and (9).

### 3. Experimental

Two samples were used to compare the planar Hall effect in Hall sensors prepared from a heterostructure with 2DEG (sample 1) and from bulk semiconductor (sample 2). For sam- ple 1 heterostructure InP/InGaAs prepared by MOCVD was used; a six arms Hall bar (Fig. 1), cca 5 mm long and $0.4\,\mathrm{mm}$  wide, was shaped by photolithography. The presence of the 2DEG in the InP/InGaAs heterostructure used for sample 1 was conﬁrmed previously by measurements of the quan- tum Hall effect at $4.2\,\mathrm{K}$  in high magnetic ﬁeld [7]. Sample 2 was made from a bulk GaAs wafer; the Hall bar was $0.5\,\mathrm{mm}$  thick,  $14\,\mathrm{mm}$  long and $2.5\,\mathrm{mm}$  wide with a contact arrange- ment like that in Fig. 1 (two contacts for the supplied current and four point contacts for the Hall effect and the resistivity measurements). In fact, we measured two more samples with thickness 7 and  $15\,\upmu\mathrm{m}$ . For the sake of clarity we show only the two ones with extreme thickness.

The basic room-temperature transport parameters of the samples are shown in Table 1;  $\mu_{\mathrm{H}}$  is the Hall mobility,  $\rho_{\mathrm{s}}$  is the sheet resistivity,  $n_{\mathrm{H,s}}$  is the sheet Hall concentration and  $U_{\mathrm{H,transv}}/(B I)$  is the sensitivity of the Hall sensor at $\theta\!=\!0$  (transversal case).

### 4. Results and discussion

Voltages  $U_{34}$  and  $U_{56}$  were measured in the dark to obtain the planar Hall coefﬁcient and the magnetoresistance param- eters, respectively. Magnetic ﬁeld of both polarities up to 1 T was applied in various directions with regard to the sen- sor plane. Supply currents of  $0.1\,\mathrm{mA}$  and  $10\,\mathrm{mA}$  were used for the heterostructure- and the bulk Hall sensors, respec- tively.

In Fig. 2, the planar Hall voltage versus the angle  $\phi$  between  $B$  and  $I$  for the 2DEG InP/InGaAs Hall sensor is plotted in the planar case  $(\theta\!=\!90^{\circ})$ ) at  $B\,{=}\,0.7$ , 0.8, 0.9 and  $1.0\,\mathrm{T}.$ . The ﬁtting curves (see Eq. (5)) for the individual magnetic ﬁelds are also shown. From the ﬁtting procedure values 2.6, 3.4, 4.3 and  $5.4\,\upmu\mathrm{V}$  were obtained for  $U_{\mathrm{H,plan,max}}$  at  $B\!=\!0.7$ , 0.8, 0.9 and  $1.0\,\mathrm{T},$  respectively. The dependence of  $U_{\mathrm{H,plan,max}}$  versus yields $B$    $U_{\mathrm{H,plan,max}}/(B^{2}I)\!=\!0.053\,\mathrm{V}\,\mathrm{T}^{-2}\,\mathrm{A}^{-1}$ which corresponds to a planar sheet Hall coefﬁcient  $R_{\mathrm{H,plan,}s}\,{=}\,0.11\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2}$  (see Eq. (6)). We observed the typical curve for the planar Hall voltage, with extrema at  $45^{\circ},\ 135^{\circ},\ 225^{\circ}$  and  $315^{\circ}$ , as it is given by Eq. (5) and shown in Fig. 2, without any anisotropy, as it was observed in Si Hall sensors [5].



InFig.3,theexperimentalresultsofthemagnetoresistance versus the angle  $\theta$  are plotted for the both samples and are ﬁtted by the following curve (10) where and $A_{1}$  are ﬁtting parameters representing the phys- $A_{2}$  ical magnetoresistance coefﬁcient  $C$  and the planar magne- toresistanceparameter $(\Delta\rho/\rho_{0})_{\mathrm{plan}}/(\mu_{\mathrm{H}}\,B)^{2}$ ,respectively.For the 2DEG InP/InGaAs sensor values 0.053 and 0.000015 were obtained for the physical magnetoresistance coefﬁcient and the planar magnetoresistance parameter, respectively. In the bulk GaAs sensor, the dependence of the magnetore- sistance parameter on the angle  $\theta$  is nearly constant and the magnetoresistance does not express signiﬁcant differ- ence between transversal and planar positions  $(A_{1}\!=\!0.018$ , ). While in the sensor from bulk semiconductor $A_{2}\,{=}\,0.015)$  we obtained $A_{1}\!\cong\!A_{2}\!\cong\!C$  $\mathrm{A}1\,{\overset{\sim}{=}}\,\mathbf{C}$ erostructure sensor exhibi $\theta$  large difference between A1 ∼= and  $\phi$  adjusted at the measurements are not exactly identi- cal with that deﬁned above. A small deviation of the starting values  $\theta_{0}$  or/and  $\phi_{0}$  from zero was accepted in the ﬁtting procedure.

 



The results obtained for InP/InGaAs heterostructure Hall sensor as well as for the bulk GaAs Hall sensor are sum- marized in Table 2. The column  $\rho_{\mathrm{s}}\ \ \mu_{\mathrm{H}}{}^{2}\,C$  represents a theoretical value for the planar Hall coefﬁcient calculated from Eq. (7) using the physical magnetoresistance coefﬁ- cient. As it can be seen, the planar sheet Hall coefﬁcient in the InP/InGaAs heterostructure sensor  $(0.11\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  is nearly three order of magnitude lower than that cor- responding to the physical magnetoresistance coefﬁcient . This is associated with a low magnetore- $(76\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  sistance in the planar case due to the extremely low thick- ness of the electrically active layer. The planar Hall sheet coefﬁcient in the bulk GaAs sensor  $(0.00\bar{36}\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$  is comparable with that corresponding to the physical magne- toresistance coefﬁcient  $(0.0058\,\mathrm{m}^{4}\,\mathrm{W}^{-1}\,\mathrm{s}^{-2})$ . The ratio of  and $U_{\mathrm{H,plan,max}}$   $U_{\mathrm{H,transv}}$  proportional to  $B$  with a constant of the proportionality  $4.2\times10^{-5}\,\mathrm{T}^{-1}$  and  $2.5\times10^{-3}\,\mathrm{T}^{-1}$  was obtained for the heterostructure- and bulk sensors, re- spectively. The unambiguos growing difference between the measured planar Hall coefﬁcient and that determined from thephysicalmagnetoresistencewithdecreasingsamplethick- ness was observed also in the samples not shown here. So, the lower the sensor thickness the greater the differ- ence between the planar Hall constants values determined from planar Hall voltage and the magnetoresistance mea- surements.



### 5. Summary

Room temperature planar Hall effect and magnetoresis- tance were measured and analysed in Hall sensors prepared from an InP/InGaAs heterostructure as well as from a bulk GaAs. Both polarities of the magnetic ﬁeld were utilized at each position of the magnetic ﬁeld adjusted with regard to the sample plane, including the transversal- and the planar positions, to obtain correct values for the planar sheet Hall coefﬁcient and the physical magnetoresistance coefﬁcient, respectively.

In the heterostructure Hall sensor, the planar sheet Hall coefﬁcient was found to be about three orders of magnitude lower than that corresponding to the theoretical value calcu- lated using the transversal magnetoresistance parameter. This is associated with extremely low thickness of the electrically active layer in these sensors. The ratio of the maximum planar Hall voltage and the maximum transversal Hall voltage was about 50 times lower in the heterostructure sensor than that in the bulk Hall sensor at the same. From this point of view $B$  Hall sensors prepared from heterostructures with 2DEG are particularly suitable to be used in systems for precise mea- surement of magnetic ﬁeld vectors.

### Acknowledgments

Authors thank S. Hasen¨ohrl for the preparing of the InP/InGaAs heterostructure, D. Greguˇsov´a for the litogra- phy shaping and M Polak for helpful discussions. This work has been supported by Grant No. 2/3115/23 from the Slovak Grant Agency VEGA.

### References

[1] C. Goldberg, R.E. Davis, New galvanomagnetic effect, Phys. Rev. 94 (1954) 1121–1125. [2] Ch. Schott, P.-A. Besse, R.S. Popovic, Planar Hall effect in the vertical Hall sensor, Sens. Actuators 85 (2000) 111–115. [3] M. Polak, I. Hlasnik, Planar Hall effect in heavy doped  $n$ -InSb and its inﬂuence on the measurement of magnetic ﬁeld components with Hall generator at  $4.2\,\mathrm{K}$ , Solid-State Electron. 13 (1970) 219– 227.



[4] K. Merinsky, Ph.D. Thesis, Institute of Electrical Engineering of the Slovak Academy of Sciences, Bratislava, 1962.

[5] F. Burger, High precision miniaturized magnetic angular encoder, Ph.D. Thesis, (EPFL No. 2269) Hartung-Gore Konstanz, Germany, ISBN: 3-89649-629-8, 2001 Series in Microsystems Vol. 11.

[6] D.C. Look, Electrical characterization of GaAs materials and devices, Wiley, Chicester, 1989, p. 61.

[7] M. Morvic, J. Betko, S. Hasen¨ohrl, D. Greguˇsov´a, V. Cambel, P. Eli´aˇs, J. Nov´ak, On quantum Hall resistance and Shubnikov de Haas effect measurements on InP/InGaAs structures, in: P. Kordoˇs, J. Nov´ak (Eds.), Proceedings of the 3rd International Workshop on Heterostruc- ture Epitaxy and Devices-HEAD’97, NATO Sciences Series 3, vol. 48, Kluwer Academic Publishers, Dordrecht, 1998, p. 285.

### Biographies

Marian Morvic and his Ph.D. from the Technical University of Bratislava (Slovakia) in 1963 and 1973, respectively. He has worked since 1962 up to now as a research and scientiﬁc worker at the Institute of Electrical Engineering of the Slovak Academy of Sciences. He worked on the preparation of III-V semiconducting materials and devices as well as on their characterization. His current research interest includes characterization of basic electrical transport parameters of semiconductor epitaxial layers.

J´ulius Betko 1960 and the CSc. (equivalent to Ph.D.) degree in Electronics in 1971, both from the Technical University, Bratislava (Slovakia). In 1992, he received the Dr.Sc. degree from the Slovak Academy of Sciences. Since 1960 he has been with Institute of Electrical Engineering of the Slovak Academy of Sciences where is working on electrical characterization of semiconductors and semiconductor structures.
## Role of brucite dissolution in calcium carbonate 

## precipitation from artificial and natural seawaters 

### Dan Nguyen Dang\*, Stéphanie Gascoin, Alaric Zanibellato, Cosmelina G. Da Silva, 

Mélanie Lemoine, Benoît Riffault†, René Sabot§, Marc Jeannin, Daniel Chateigner, Otavio 

 $G i l^{j^{\mathrm{\ell}}}$ 

†Normandie Université, UNICAEN, ABTE, 14000 Caen, France 

‡ Normandie Université, ENSICAEN, UNICAEN, CNRS, CRISMAT, 14000 Caen, France 

§Université de La Rochelle, CNRS, LaSIE, 17000 La Rochelle, France 

PHREEQC computer program was used to simulate brucite equilibrium in pure water and in  different seawaters (NSW, ASW, ASW+WSM, ASW- $\cdot\mathrm{Mg}^{2+}$  at room temperature  $(20^{\circ}\mathrm{C})$ . The  modelling was carried out by adding brucite in several steps to the solution (1kg) until  equilibrium with brucite was reached. The llnl.dat database was used for all simulations.  

The saturation index (SI) is defined as: 

 $\mathrm{SI}=\log\,\mathrm{IAP}-\log\,\mathrm{Ksp}$  where IAP is ionic activity product and Ksp is solubility product of  the given ionic compound. 

The solution pH and the saturation indices for brucite, aragonite, calcite, dolomite, magnesite,  and monohydrocalcite were calculated. 

  

   
## Role of brucite dissolution in calcium carbonate 

## precipitation from artificial and natural seawaters 

### Dan Nguyen Dang\*, Stéphanie Gascoin, Alaric Zanibellato, Cosmelina G. Da Silva, 

Mélanie Lemoine, Benoît Riffault†, René Sabot§, Marc Jeannin, Daniel Chateigner, Otavio 

 $G i l^{j^{\mathrm{\ell}}}$ 

†Normandie Université, UNICAEN, ABTE, 14000 Caen, France 

‡ Normandie Université, ENSICAEN, UNICAEN, CNRS, CRISMAT, 14000 Caen, France 

§Université de La Rochelle, CNRS, LaSIE, 17000 La Rochelle, France 

PHREEQC computer program was used to simulate brucite equilibrium in pure water and in  different seawaters (NSW, ASW, ASW+WSM, ASW- $\cdot\mathrm{Mg}^{2+}$  at room temperature  $(20^{\circ}\mathrm{C})$ . The  modelling was carried out by adding brucite in several steps to the solution (1kg) until  equilibrium with brucite was reached. The llnl.dat database was used for all simulations.  

The saturation index (SI) is defined as: 

 $\mathrm{SI}=\log\,\mathrm{IAP}-\log\,\mathrm{Ksp}$  where IAP is ionic activity product and Ksp is solubility product of  the given ionic compound. 

The solution pH and the saturation indices for brucite, aragonite, calcite, dolomite, magnesite,  and monohydrocalcite were calculated. 

  

   
# Estimating Reactivity Ratios From Triad Fraction Data

Emma Hauch, Xiaoqin Zhou, Thomas A. Duever,\*Alexander Penlidis Summary: Reactivity ratio estimation is a non-linear estimation problem. Typically, reactivity ratios are estimated using the instantaneous copolymer composition equation, otherwise known as the Mayo-Lewis model, based on low conversion  copolymer composition data. However, there are other instantaneous models, $(<\!5\%)$  which can be used to estimate reactivity ratios, such as the instantaneous triad fraction equations. The aim of this paper is to determine the potential improvement in reactivity ratio estimates when triad fraction data is used in place of and in combination with copolymer composition data. The interest in using triad fraction data in parameter estimation, stems from the fact that there are a greater number of responses measured (six triad fractions) compared to composition leading to data with theoretically more information content. In principle this should lead to reactivity ratio estimates having less uncertainty. In this study, the parameter estimates are obtained by employing the error in variables model (EVM), assuming a multiplicative error structure. Several case studies involving published literature data for different copolymer systems are presented. As the case studies demonstrate in general more precise estimates can be obtained from triad fraction data. Combining the triad fraction with composition data leads to little additional improvement. However, discrepancies arise between reactivity ratios estimated from composition data compared with those obtained from triad fraction data depending upon the copolymer system. Those copolymer systems exhibiting more heterogeneity due to phase separation during polymerization may be showing more discrepancy.



### Keywords: copolymerization; error-in-variables model; radical polymerization; reactivity ratios; triad fractions

## Introduction

The justiﬁcation of using the error in variables model (EVM) approach for para- meter estimation, especially with respect to the reactivity ratio estimation problem in copolymerization, has been demonstrated extensively in the literature in the last twenty years or so. In the EVM frame- work, all variables (dependent and inde- pendent) are considered to be subject to error.

Typically, copolymerization reactivity ratios are estimated using the instantaneous copolymer composition equation, based on low conversion  $(<\!5\%)$  copolymer composi- tion data. Compared to methods using copolymer composition data, efforts to directly use triad fraction (sequence length) data for reactivity ratio estimation are far fewer. In part this is explained by the fact that it is more difﬁcult to obtain triad fractions data. Table 1 gives a summary of a recent survey of the literature of papers that report triad fraction data.



In this paper case studies are reported forthesystemsacylamide/acylonitrile (AN), AN/acrylic acid (AA), AN/ pentyl-methacrylate (PMA), and ﬁnally for the system styrene (STY)/AN, and STY/ AN in toluene, Reactivity ratios are estimated using composition data, triad fraction data and a combination of the two.

Macromol. Symp. 2008, 271, 48–6349

## Copolymer Reactivity Ratio Estimation

Obtaining reactivity ratios from triad frac- tion data is a non-linear multi-response parameter estimation problem. Six triad fractions provide, in principle, more infor- mationthandoesasinglecopolymer composition measurement; hence the extra information should result in greater preci- sion of the reactivity ratio estimates. Triad fractions are sequences of three consecu- tive monomer units in a copolymer chain. The fractions are categorized as either monomer-1-centered triads or monomer- 2-centered triads. The triads are denoted by $\mathbf{A_{ijk}}$ , where i, j and $\mathbf{k}$  are either monomer units 1 or 2. The six triad fractions can be related to the reactivity ratios through the following equations: fraction from each set thus leading to an estimation scheme in which the reactivity (1) where  $\mathrm{f}_{1}$  and  $\mathrm{f}_{2}$  represent the mole fraction of monomer 1 and 2 in the feed, respectively, and $\mathbf{r}_{1}$  and $\mathbf{r}_{2}$  are the reactivity ratios. In addition the sum of the monomer- 1-centered triad fractions is unity and similarlywiththemonomer-2-centered fractions. There are therefore two linear dependencies in the data. Linear or near- linear dependencies in the data will lead to unstable parameter estimates. A solution to the problem is to eliminate one triad Abbrevations of monomers/chemicals.

     

    

   

  

 



ratios are estimated from four triad frac- tions. As a result, in our work  $\mathrm{A}_{111}$  and  $\mathrm{A}_{222}$  were eliminated in order to avoid the issue of linear dependency.

The instantaneous copolymer composi- tion model is often referred to as the Mayo- Lewis model. Copolymer composition is the mole fraction of monomer 1 incorpo- rated in the copolymer. Therefore, the Mayo-Lewis model relates the monomer feedcompositionwiththecopolymer composition, $\mathbf{F}_{1}$ , via the following equation:


$$
\mathrm{F_{1}}=\frac{\mathrm{r_{1}f_{1}^{2}}+\mathrm{f_{1}}(1-\mathrm{f_{1}})}{\mathrm{r_{1}f_{1}^{2}}+2\mathrm{f_{1}}(1-\mathrm{f_{1}})+\mathrm{r_{2}}{\left(1-\mathrm{f_{1}}\right)}^{2}}
$$


(2) The implementation of the above equa- tions in the parameter estimation scheme considered a multiplicative error structure based on previous experience. Further- more, the error associated with the feed composition was assumed to be  $\pm1\,\%$  error for triad fractions was assumed to be  $\pm10\%$  composition was assumed to be  $\pm5\%$ .



## Error-in-variables Model (EVM)

Theerror-in-variablesmodel(EVM) method is a relatively recent statistical approach to solving multi-response para- meterestimationproblems,wherethe errors in all measured variables are taken into account. The approach is particularly applicable for the estimation of copolymer reactivity ratios because all of the data, on which the estimation is based, are measure- ments, which contain errors, and some of the errors can be large. Another beneﬁt of using the EVM method is that the algo- rithm yields not only estimates of the true parameter values, but also estimates of the true values of the measurements. Addi- tionally, EVM allows the user to assign relative weights to the measured quantities according to their precision, via the input of a measurement covariance matrix. The parameter estimation method also allows conﬁdence intervals to be calculated for the estimated parameters.



The problem is described in detail and algorithms for parameter estimation are given by Reilly and Patino-Leal, while Patino-Leal et al. and Rossignoli and Duever have described in detail the application of EVM to the estimation of reactivity ratios in copolymer systems. An abbreviated version of the approach is presented below.

The EVM model consists of two state- ments. First, the vxl vector of measure- ments $\mathrm{\underline{{X}}_{i}}$  is equated to the vxl vector of true values, at the $\xi_{\mathrm{i}}.$  $\mathrm{i}^{\mathrm{th}}$  of n conditions, plus an additive error $\varepsilon_{\mathrm{i}}$ 


$$
\underline{{\mathbf{X}}}_{\underline{{\mathbf{i}}}}=\underline{{\boldsymbol{\xi}}}_{\underline{{\mathbf{i}}}}+\underline{{\boldsymbol{\varepsilon}}}_{\underline{{\mathbf{i}}}}\quad\mathrm{where}\quad\mathrm{i}=1,2,....,\boldsymbol{\mathsf{n}}
$$


(3) Here, the elements of $\underline{{\mathrm{X}}}\mathrm{i}$  are the measured values of feed composition, triad fractions and copolymer composition. The error vector  $\varepsilon_{\mathrm{i}}$  is often assumed to be normally distributed with a mean vector of 0, and a vxv covariance matrix  $\underline{{\mathrm{V}}}.$ , which is non- singular and may be known or unknown Since the errors of the measurements used in this work are assumed to be multi- plicative in nature, a logarithmic transfor- mation is applied resulting in

 




$$
\ln(\underline{{x}}_{i})=\ln(\underline{{\xi}}_{i})+\varepsilon_{i}
$$


(4) Thesecondstatementspeciﬁesthe model, which relates the true values of the parameters and variables,




$$
\underline{{\mathrm{f}}}(\underline{{\xi}}_{\underline{{\mathrm{i}}}},\underline{{\theta}}^{*})=0\quad\mathrm{where}\quad\mathrm{i}=1,2.\ .\ .\ .,\mathsf{n}
$$


The vector of functions f in Equation (4) is mxl, and may be linear or nonlinear in the elements of and $\xi_{\mathrm{i}}$  u. The vector contains $\underline{{\theta}}$  the reactivity ratios to be estimated. Here the functions are the triad fraction and composition models speciﬁed in Equations (1) and (2). Using a Bayesian approach, that a point estimate  $\underline{{\hat{\theta}}},$ , can be found by minimizing





(6) where  $r_{i}$  is the number of replicates at the $\mathrm{i}^{\mathrm{th}}$  trial.



The optimization algorithm uses a New- ton’s method similar to Fisher’s method of scoring to minimize  $\Phi$ . The routine to solve the EVM problem uses a nested- iterative scheme with an inner iteration that estimates the true values of the variables,, while keeping the parameter values con- stant, and an outer iteration that estimates the true values of the parameters,, while keeping the variables,, constant. The algorithm starts by using the initial para- meter estimates (supplied by the user) and the  $\boldsymbol{\upxi}^{0}$ set equal to the measured variables $\mathrm{\underline{{X}}_{i}}$ . Theinnerschemeusesthefollowing equation iteratively to update the estimates of the variables,


$$
\begin{array}{c}{{\displaystyle{\underline{{\xi}}_{i}}^{(k+1)}=\overline{{\underline{{x}}}}_{i}-{\underline{{V B^{\prime}}}}_{i}\,\left({\underline{{B}}}_{i}\,{\underline{{V B^{\prime}}}}_{i}\right)^{-1}}}\\ {{\displaystyle\qquad\times\left[f\big(\underline{{\xi}}^{(k)},\underline{{\theta}}\big)\right)+{\underline{{B}}}_{i}\big({\underline{{x}}}_{i}-\underline{{\xi}}^{(k)}\big)\big]}}\end{array
$$


(7) 
$$
\phi=\,\frac{1}{2}\sum_{i=1}^{n}r_{i}\,\left(\underline{{{x}}}_{i}-\ \underline{{{\hat{\xi}}}}_{i}\,\right)^{T}\,\underline{{{V}}}^{-1}\!\left(\underline{{{x}}}_{i}-\ \underline{{{\hat{\xi}}}}_{i}\right)
$$


where  $\underline{{\mathbf{B}}}_{\mathrm{i}}$  is the vxl vector of partial derivatives of the function with respect to the variables,


$$
B_{i}=\left[\frac{d f\,\left(\underline{{\xi}}_{i}\,,\underline{{\theta}}\right)}{d\,\left(\underline{{\xi}}_{i}\right)_{t}}\right]_{\underline{{\xi}}_{i}=\ \underline{{\xi}}_{i}(k)}
$$


(8)

### for the  $t^{t h}$ element

and $\underline{{\mathrm{V}}}$  is the vxv error covariance matrix for the measurements. Note that V must either besupplied bythe useror must be estimated from sufﬁcient replicates.

The outer scheme uses the following equation iteratively to update the estimates of the parameters,


$$
\underline{{\theta}}^{(\mathsf{u+1})}=\underline{{\theta}}^{(\mathsf{u})}-\underline{{\mathbf{G}}}^{-1}\underline{{\mathbf{q}}}
$$


(9) where  $\underline{{\mathrm{G}}}$  is the pxp expected information matrix given by,




$$
\begin{array}{r l}{G=E\left[{\frac{d^{2}\phi}{d\theta_{i}d\theta_{j}}}\right]\;}&{{}i,j\;e l e m e n t}\\ {=}&{{}\displaystyle\sum_{i=1}^{n}\;r_{i}\;\underline{{Z}}_{i}^{\prime}\;(\underline{{B}}_{i}\;\underline{{V}}B_{i}^{\prime})^{-1}\underline{{Z}}_{i}}\end{array}
$$


(10) and  $\mathtt{q}$  is the pxl gradient vector





and $Z_{\mathrm{i}}$  is the lxp vector of partial derivatives with respect to the parameters,



(12) The only assumption required for EVM is that of model adequacy, while the construction ofthe conﬁdenceregions requires that the errors are independent and identically normally distributed. Gen- erally, to quantify the uncertainty in the parameter estimates the  $95\%$  joint prob- ability contour is determined via either of the following two methods. The ﬁrst is a relatively rough approximation to the joint posterior probability region for the esti- mated parameters, and is given by




$$
(\underline{{\theta}}-\underline{{\hat{\theta}}})^{\prime}\;G(\underline{{\theta}}-\underline{{\hat{\theta}}})\;\leq\;\chi_{(p,1-\alpha)}^{2}
$$


(13) where the ‘‘ $\cdot\wedge\cdot$  indicates estimates of the parameters, G is the expected information matrix of the parameter estimates, and  $\chi^{2}$   $(\mathrm{p},\ \ 1-\upalpha)$  l  a) is the value of the chi-squared distribution having  $\mathfrak{p}$  degrees of freedom, exceeded with probability  $\alpha.^{[1]}$  Here the assumption of multivariate normality is made and the elliptical posterior probabil- ity contour is determined. It is important to note that both the shape and probability content of this contour are approximate.

 



The second method is a somewhat better alternativetoapproximatingthejoint posterior probability region (JPR), and is given by the equation


$$
\phi\big(\underline{{{\theta}}}\big)=\phi\big(\underline{{{\hat{\theta}}}}\big)+\frac{1}{2}\,\chi_{(p\mathrm{~,~}1-\alpha)}^{2}
$$


(14) 
$$
\begin{array}{r l}{\underline{{q}}=\left[\frac{d\phi}{d\theta_{j}}\right]}&{{}i^{t h}\;e l e m e n t}\\ {=}&{{}\sum_{i=1}^{n}r_{i}\;\underline{{Z^{\prime}}}_{i}(\underline{{B}}_{i}\;\underline{{V B^{\prime}}}_{i})^{-1}\underline{{B}}_{i}(\underline{{\overline{{x}}}}_{i}-\underline{{\hat{\xi}}}_{i})}\end{array}
$$


where  $\phi$  is given by Equation (6). This approximation gives contours of the correct 
$$
\underline{{\underline{{Z_{i}}}}}=\left[\frac{d f(\underline{{\xi}}_{i},\:\theta)}{d\theta_{k}}\right]\qquad\quad k^{t h}\;e l e m e n t
$$


shape (not a general elliptical shape) but of approximateprobabilitycontent.All contours shown in this paper are calculated using equation (14).

## Results and Discussion

The following case studies were analyzed in order to: (1) comparereactivity ratio estimates obtained from composition data with those calculated from triad fraction data,(2)determineiftriadfractions resulted in less uncertainty in the parameter estimates; and (3) evaluate the potential improvement in the estimates when the triad fractions and composition are com- bined.

Copolymer Systems Case I: Acrylamide/Acrylonitrile Mukherjee et al. studied the acryla- minde/acrylonitrile (AN) system at $65\,^{\circ}\mathrm{C}$  in bulkcopolymerizationandpresented experimental data for both copolymer composition and triad fractions. The JPR’s for the different data types can be seen in Figure 1. The JPR based on composition data is much larger than the JPR obtained when using triad fraction data. Further- more, the JPR obtained from triad fraction analysis, falls within the JPR obtained from analysis of the composition data. For this system composition data and triad fraction data lead to consistent reactivity ratio estimates and triad fraction data leads to estimates with a smaller amount of uncer- tainty as quantiﬁed by the areas of the JPR. Diagnostic checking was performed in order to determine whether the reactivity ratio estimates in fact model the experi- mental results accurately. The reactivity ratio estimates, obtained from analyzing the triad fractions and composition data separately, were used in the triad fraction equations (Eq. 1) to compare predicted triad fractions with the published experi- mental triad fractions. It can be seen in Figures 2 a) – d) that the predicted triad fractions,calculatedusingeitherthe composition-based or the triad fraction- based reactivity ratio estimates, are in good agreement with the experimental triad fractions results. Furthermore, the copoly- mer composition can be predicted well with reactivity ratios that are estimated from either composition data or triad fraction data, as seen in Figure  $2\mathrm{~e~}$ ). These results conﬁrm that the estimates obtained from either data type are consistent.

  

 



Figure 3 shows the JPR obtained by combining triad fraction with composition dataandcomparesitwiththeJPR calculated using only triad fractions. This plot was produced to determine if combin- ing triad fractions with composition would produce even better parameter estimates. It can be seen in Figure 3 that the use of composition data in conjunction with the triad fractions data does not result in a signiﬁcant decrease in uncertainty, com- pared to simply using triad fraction data. Subsequently, for this system, on might argue that copolymer composition data does not contribute signiﬁcant additional information to the estimation of reactivity ratios, given that triad fraction data is readily available.

 



Case II: Acrylonitrile/Acrylic Acid Brar and Dutta studied the acrylonitrile (AN)/acrylic acid (AA) system at  $27^{\circ}$  in bulk photo-initiated copolymerization and presented experimental data on both copo- lymer composition and triad fractions. The data are shown in Table 2.

Note that a total of 10 experiments are available in which copolymer composition data was measured, eight in which triad fractions were measured and only six for which both composition and triad fractions are available. The different number of data points and the difference in the feed compositions at which data are available complicate the case study somewhat. In the article the composition data was analyzed to obtain reactivity ratio point estimates, which were then used to obtain predicted values for the triad fractions. Model predic- tions of triad fractions based on composition- based reactivity ratio estimates were found to be in good agreement with the experi- mentally determined triad fractions. Here the consistency between the copolymer composition and triad fractions is explored by comparing the resulting reactivity ratio estimates for the two different measure- ments. From Figure 4 it can be seen that the JPR of the triad fraction analysis is smaller in area than the composition JPR showing again that the use of triad fraction data results in more precise reactivity ratio estimates, compared with those obtained from composition data. In comparison to the previous example the two JPRs do not overlapasmuch.Diagnosticchecking performed for the AN/AA system showed exactlythesameresultsaswiththe diagnostic checking for the previous case study. In other words, when the composition- based and triad fraction-based estimates were used to predict the experimental results good agreement was again found. Therefore for this system, the estimates are again consistent.



Figure 5 shows the comparison between the results obtained when combining triad fractions with composition and just using triad fractions. Again the regions are of approximately the same size, indicating no signiﬁcant difference in parameter preci- sion. Note that the combined data contour lies above that for triad fraction data only. The latter in turn lies above that based on composition data (Figure 4). Normally one would expect the combined contour to fall somewhere between the composition- and triad fraction-based contours. The explana- tion here is related to the differences in feed compositions between the different data sets.

Case Study III: Acrylonitrile/Pentyl Methacrylate Brar and Dutta also studied the acryloni- trile (AN)/ pentyl methacrylate (PMA) system at room temperature in bulk and presented experimental data on both copo- lymer composition and triad fractions.



Figure 6 shows the JPR from the EVM analysis of the composition data to be much larger than the JPR from the EVM analysis of the triad fraction data. That is, this case study is further testimony to the fact that the use of triad fraction data results in less uncertainty in the reactivity ratio para- meter estimates. In this case, however, there is a considerable difference in the estimates obtained, in that the composition- based estimates are considerably larger.

 



Similar to the procedure in the other case studies, diagnostic checking was then performed to ensure the ﬁt of the model to the experiment results. Figures  $^{7\mathrm{a-d}}$  show the predicted triad fractions obtained when using the composition-based and triad fraction-based reactivity ratio estimates, while Figure 7 e shows the copolymer com- position predicted from the Mayo-Lewis equation. In this case, triad fractions are best predicted using triad-based reactivity ratio estimates. Composition-based esti- mates do not yield very good triad fraction predictions and triad fraction-based esti- mates do not yield very good copolymer composition predictions. In addition, triad fraction A121 is not predicted well by either set of reactivity ratios. There is an observed lack of ﬁt present. One possible explanation might be related to the calculation required to convert the actual measurements, namely the NMR normalized peak areas, into triad fractions. These calculations require addi- tional parameters whose values might be inconsistent for this triad fraction. The peak areas and the converting equations are unfortunately not reported. Hence the source for the lack of ﬁt is unknown.



Finally, combining copolymer composi- tion and triad fraction data showed the same trends as seen in previous examples.

Case Study IV: Styrene/Acrylonitrile A number of groups have published work on the styrene (STY) – acrylonitrile (AN) system for both bulk and solution copoly- merization. Here we analyzed data from the groups of Hill et al. and Klumper- man and Kraeger. Data is reported for both bulk and in toluene solvent.

A clearer picture of the estimates from the different data types can be seen when the data is grouped together by system (i.e., bulk and toluene solvent) and the JPR’s are shown. Note that in the ﬁgures of this case study, if the data set used in the analysis is taken from Hill et al, Hill et al. or Klumperman and Kraeger, it is referred to in the ﬁgure as (Hill, 1989), (Hill, 1992) or(Klumperman,1994),respectively. Furthermore, in all cases monomer 1 and 2 refer to styrene and acrylonitrile mono- mer, respectively.



In Figure 8 the results are shown for the Hill et al. triad fractions and composi- tion data sets for bulk copolymerizations. This plot indicates relatively large differ- ences between composition-based results and triad fraction-based estimates similar to the results observed for the AN/PMA system. The two different triad fraction data sets yield results which are relatively close. Furthermore, by considering Figure 9 with the bulk STY/AN studies from Hill et al. and Klumperman and Kraeger, it can be seen that the regions from the different data sources do not overlap, for neither composition nor triad fraction data.

 



Similar trends are again seen when consideringtheSTY/ANstudiesin toluene (see Figure 10). Note however that the joint probability regions are closer together than they are in the analysis of the bulk experiments.



A possible explanation for the trends observed over the four different copolymer systems may have to do with the varying degrees of heterogeneity (preferential sorp- tion or solvent effects) exhibited due to phase separation or precipitation during copolymerization. These ‘‘heterogeneity’’ effects became progressively more signiﬁ- cant as we moved from case study I to IV. The effects have been documented before by several researchers for several copoly- mer systems involving AN and carboxylic acids. An increasing degree of hetero- geneity leads to increased difﬁculty in obtaining polymer samples that give repre- sentative triad fraction measurements that are consistent with composition data. This may explain why reactivity ratio estimates from triad fractions on the one hand and composition on the other compare favour- ably for the acrylamide/AN and the AN/ AA systems, but do not for the AN/PMA and STY/AN systems. Further support for this hypothesis is seen in Figure 10, where the results are more consistent when STY/ AN is copolymerized in toluene solvent, which will make the system less hetero- geneous.



## Concluding Remarks

Thispaperexaminedthequestionof whether or not triad fraction data can lead NOTE:  $\mathsf{\Delta}=\mathsf{Y e s},\,\mathsf{X}\!=\!\mathsf{N o}.$  $\surd=$  The following abbreviations are used: Klump $=$ Klumperman and Kraeger; Hill ¼ Hill et al.. Blank spaces in the table indicate that the analysis was not performed for that particular entry.



to reliable reactivity ratio estimates. Sev- eral aspects were investigated, namely: (1) Are triad fraction-based estimates consis- tent with composition-based estimates, (2) Do triad fractions lead to reactivity ratio estimates with less uncertainty, and (3) Is there and advantage to combining triad fraction and copolymer composition data. These questions were explored using four different case studies involving low con- version data from four different copolymer systems. Table 3 summarizes the results of these investigations.

In general triad fraction data did con- sistently result reactivity ratio estimates havingsigniﬁcantlysmalleruncertainty compared to those obtained from composi- tion data. Combining triad fractions with copolymer composition did not reduce the uncertainty in the reactivity ratio estimates (therefore leading to higher quality esti- mates), compared to those obtained only from triad fractions. The reactivity ratio estimates may not agree for the same system, when different data types are used. That is, the parameter estimates obtained from analyzing triad fraction data are not always in agreement with the estimates obtained from analyzing composition data. Heterogeneityinthecopolymerization process may provide an explanation for this observation.

Acknowledgements: Financial support from the Natural Science and Engineering Research Council of Canada (NSERC), the Canada Research Chair (CRC) program, and a Province of Ontario Graduate Scholarship (OGS) is gratefully acknowledged.

[1] A. L. Polic, T. Duever, A. Penlidis, J. Poly. Sci. Polym. Chem. 1994, 36, 813.

[2] P. Rossignoli, T. Duever, Polym. React. Eng. 1995, 3, 361.

[3] M. Dube, R. Sanayei, A. Penlidis, K. O’Driscoll, P. Reilly, J. Polym. Sci., Polym. Chem. 1991, 29, 703.

[4] A. Brar, K. Dutta, J. Appl. Polym. Sci. 1998, 69, 2507. [5] A. Brar, K. Dutta, Eur. Polym. J. 1998, 34, 1585.

[6] A. Brar, B. Jayaram, K. Dutta, J. Polym. Mater. 1994, 11, 171.

[7] A. Brar, M. Kaur, J. Appl. Polym. Sci. 2003, 88, 3005. [8] A. Brar, M. Kaur, Eur. Polym. J. 2003, 39, 705.

[9] A. Brar, D. Pradhan, J. Appl. Polym. Sci. 2003, 89, 1779.

[10] A. Brar, D. Pradhan, J. Mol. Struct. 2003, 649, 245. [11] A. Brar, R. Kumar, M. Kaur, J. Mol. Struct. 2003, 650, 85.

[12] P. Brown, K. Fujimori, Polymer 1994, 35, 2302. [13] A. Burke, T. Duever, A. Penlidis, Macromol. Theory Simul. 1994, 3, 1005.

[14] J. de la Fuente, M. Fernandez-Garcia, M. Fernandez- Sanz, E. Madruga, Macromolecules 2001, 34, 5833. [15] I. Erol, F. Yavuz, M. Durgun, Polym. J. 2004, 36, 303.

[16] F. Escher, G. Galland, J. Polym. Sci., Polym. Chem. 2004, 42, 2474.

[17] A. Ferando, A. Longo, Polym. Prepr. 1997, 38, 798. [18] C. Fernandez-Monreal, M. Sanchez-Chaves, G. Martinez, E. Madruga, Acta Polym. 1999, 50, 408.

[19] M. Galimberti, F. Piemontesi, O. Fusco, I. Camurati, M. Destro, Macromolecules 1998, 31, 3409.

[20] N. Ha, Comp. Theor. Polym. Sci. 1998, 7, 121.

[21] T. Ha, K. Fujimori, D. Tucker, P. Henry, J. Macromol. Sci., Pure Appl. Chem. 1999, A36, 893.

[22] D. Hill, J. O’Donnell, P. O’Sullivan, Eur. Polym. J. 1989, 25, 911.

[23] D. Hill, A. Lang, P. Munro, J. O’Donnell,Eur. Polym. J. 1992, 28, 391.

[24] S. Hooda, A. Brar, J. Appl. Polym. Sci. 2003, 88, 3232.

[25] Y. Kim, H. Harwood, Polymer 2002, 43, 3229.

[26] Y. Kim, D. Priddy, H. Harwood, Polymer 2003, 44, 4165.

[27] B. Klumperman, I. Kraeger, Macromolecules 1994, 27, 1529.

[28] I. Kramer, H. Pasch, H. Handel, K. Albert, Macro- mol. Chem. Phys. 1999, 200, 1734.

[29] I. Maxwell, A. Aerdts, A. German, Macromolecules 1993, 26, 1956.

[30] M. Mukerjee, S. Chatterjee, A. Brar, J. Appl. Polym. Sci. 1999, 73, 55.

[31] K. O’Leary, D. Paul, Polymer, 2004, 45, 6575.

[32] P.Pazhanisamy,M.Ariff,Q.Anwaruddin, J. Macromol. Sci., Pure Appl. Chem. 1997, A34, 1045. [33] A. Rudin, K. O’Driscoll, M. Rumack, Polymer 1981, 22, 740.

[34] C. Selvamalar, P. Vijayanand, A. Penlidis, S. Nanjundan, J. Appl. Polym. Sci. 2004, 91, 3604.

[35] C. Soyhan, I. Erol, J. Appl. Polym. Sci. 2004, 91, 964. [36] S. Sundarrajan, K. Ganesh, K. Srinivasan, Polymer, 2003, 44, 61.

[37] S. Wharry, Polymer 2004, 45, 2985.

[38] Y. Zhao, P. Liu, H. Liu, J. Jiang, C. Chen, F. Xi, Macromol Rapid. Commun. 2001, 22, 633.

[39] J. Koenig, ‘‘Chemical microstructure of polymer chains’’, John Wiley & Sons Ltd, New York 1980.

[40] P. Reilly, H. Patino-Leal, Technometrics 1981, 23, 221.

[41] H. Patino-Leal, P. Reilly, K. O’Driscoll, J. Polym. Sci. Polym. Lett. 1980, 18, 219.

Macromol. Symp. 2008, 271, 48–6363

[42] S. Keeler, P. Reilly, Can. J. Chem. Eng. 1991, 69, 27. [43] P. Reilly, S. Keeler, H. Reilly, Appl. Stat. 1993, 42, 693.

[44] L. Garcia-Rubio, A. Hamielec, J. Appl. Polym. Sci. 1979, 23, 1397.

[45] L.Garcia-Rubio,G.Lord,J.Macgregor,A. Hamielec, Polymer 1985, 26, 2001.

[46] K. Plochocka, H. Harwood, Polym. Prepr. 1978, 19, 240.

[47] Y. Semchikov, Macromol. Symp. 1996, 111, 317.
# Estimating Reactivity Ratios From Triad Fraction Data

Emma Hauch, Xiaoqin Zhou, Thomas A. Duever,\*Alexander Penlidis Summary: Reactivity ratio estimation is a non-linear estimation problem. Typically, reactivity ratios are estimated using the instantaneous copolymer composition equation, otherwise known as the Mayo-Lewis model, based on low conversion  copolymer composition data. However, there are other instantaneous models, $(<\!5\%)$  which can be used to estimate reactivity ratios, such as the instantaneous triad fraction equations. The aim of this paper is to determine the potential improvement in reactivity ratio estimates when triad fraction data is used in place of and in combination with copolymer composition data. The interest in using triad fraction data in parameter estimation, stems from the fact that there are a greater number of responses measured (six triad fractions) compared to composition leading to data with theoretically more information content. In principle this should lead to reactivity ratio estimates having less uncertainty. In this study, the parameter estimates are obtained by employing the error in variables model (EVM), assuming a multiplicative error structure. Several case studies involving published literature data for different copolymer systems are presented. As the case studies demonstrate in general more precise estimates can be obtained from triad fraction data. Combining the triad fraction with composition data leads to little additional improvement. However, discrepancies arise between reactivity ratios estimated from composition data compared with those obtained from triad fraction data depending upon the copolymer system. Those copolymer systems exhibiting more heterogeneity due to phase separation during polymerization may be showing more discrepancy.



### Keywords: copolymerization; error-in-variables model; radical polymerization; reactivity ratios; triad fractions

## Introduction

The justiﬁcation of using the error in variables model (EVM) approach for para- meter estimation, especially with respect to the reactivity ratio estimation problem in copolymerization, has been demonstrated extensively in the literature in the last twenty years or so. In the EVM frame- work, all variables (dependent and inde- pendent) are considered to be subject to error.

Typically, copolymerization reactivity ratios are estimated using the instantaneous copolymer composition equation, based on low conversion  $(<\!5\%)$  copolymer composi- tion data. Compared to methods using copolymer composition data, efforts to directly use triad fraction (sequence length) data for reactivity ratio estimation are far fewer. In part this is explained by the fact that it is more difﬁcult to obtain triad fractions data. Table 1 gives a summary of a recent survey of the literature of papers that report triad fraction data.



In this paper case studies are reported forthesystemsacylamide/acylonitrile (AN), AN/acrylic acid (AA), AN/ pentyl-methacrylate (PMA), and ﬁnally for the system styrene (STY)/AN, and STY/ AN in toluene, Reactivity ratios are estimated using composition data, triad fraction data and a combination of the two.

Macromol. Symp. 2008, 271, 48–6349

## Copolymer Reactivity Ratio Estimation

Obtaining reactivity ratios from triad frac- tion data is a non-linear multi-response parameter estimation problem. Six triad fractions provide, in principle, more infor- mationthandoesasinglecopolymer composition measurement; hence the extra information should result in greater preci- sion of the reactivity ratio estimates. Triad fractions are sequences of three consecu- tive monomer units in a copolymer chain. The fractions are categorized as either monomer-1-centered triads or monomer- 2-centered triads. The triads are denoted by $\mathbf{A_{ijk}}$ , where i, j and $\mathbf{k}$  are either monomer units 1 or 2. The six triad fractions can be related to the reactivity ratios through the following equations: fraction from each set thus leading to an estimation scheme in which the reactivity (1) where  $\mathrm{f}_{1}$  and  $\mathrm{f}_{2}$  represent the mole fraction of monomer 1 and 2 in the feed, respectively, and $\mathbf{r}_{1}$  and $\mathbf{r}_{2}$  are the reactivity ratios. In addition the sum of the monomer- 1-centered triad fractions is unity and similarlywiththemonomer-2-centered fractions. There are therefore two linear dependencies in the data. Linear or near- linear dependencies in the data will lead to unstable parameter estimates. A solution to the problem is to eliminate one triad Abbrevations of monomers/chemicals.

     

    

   

  

 



ratios are estimated from four triad frac- tions. As a result, in our work  $\mathrm{A}_{111}$  and  $\mathrm{A}_{222}$  were eliminated in order to avoid the issue of linear dependency.

The instantaneous copolymer composi- tion model is often referred to as the Mayo- Lewis model. Copolymer composition is the mole fraction of monomer 1 incorpo- rated in the copolymer. Therefore, the Mayo-Lewis model relates the monomer feedcompositionwiththecopolymer composition, $\mathbf{F}_{1}$ , via the following equation:


$$
\mathrm{F_{1}}=\frac{\mathrm{r_{1}f_{1}^{2}}+\mathrm{f_{1}}(1-\mathrm{f_{1}})}{\mathrm{r_{1}f_{1}^{2}}+2\mathrm{f_{1}}(1-\mathrm{f_{1}})+\mathrm{r_{2}}{\left(1-\mathrm{f_{1}}\right)}^{2}}
$$


(2) The implementation of the above equa- tions in the parameter estimation scheme considered a multiplicative error structure based on previous experience. Further- more, the error associated with the feed composition was assumed to be  $\pm1\,\%$  error for triad fractions was assumed to be  $\pm10\%$  composition was assumed to be  $\pm5\%$ .



## Error-in-variables Model (EVM)

Theerror-in-variablesmodel(EVM) method is a relatively recent statistical approach to solving multi-response para- meterestimationproblems,wherethe errors in all measured variables are taken into account. The approach is particularly applicable for the estimation of copolymer reactivity ratios because all of the data, on which the estimation is based, are measure- ments, which contain errors, and some of the errors can be large. Another beneﬁt of using the EVM method is that the algo- rithm yields not only estimates of the true parameter values, but also estimates of the true values of the measurements. Addi- tionally, EVM allows the user to assign relative weights to the measured quantities according to their precision, via the input of a measurement covariance matrix. The parameter estimation method also allows conﬁdence intervals to be calculated for the estimated parameters.



The problem is described in detail and algorithms for parameter estimation are given by Reilly and Patino-Leal, while Patino-Leal et al. and Rossignoli and Duever have described in detail the application of EVM to the estimation of reactivity ratios in copolymer systems. An abbreviated version of the approach is presented below.

The EVM model consists of two state- ments. First, the vxl vector of measure- ments $\mathrm{\underline{{X}}_{i}}$  is equated to the vxl vector of true values, at the $\xi_{\mathrm{i}}.$  $\mathrm{i}^{\mathrm{th}}$  of n conditions, plus an additive error $\varepsilon_{\mathrm{i}}$ 


$$
\underline{{\mathbf{X}}}_{\underline{{\mathbf{i}}}}=\underline{{\boldsymbol{\xi}}}_{\underline{{\mathbf{i}}}}+\underline{{\boldsymbol{\varepsilon}}}_{\underline{{\mathbf{i}}}}\quad\mathrm{where}\quad\mathrm{i}=1,2,....,\boldsymbol{\mathsf{n}}
$$


(3) Here, the elements of $\underline{{\mathrm{X}}}\mathrm{i}$  are the measured values of feed composition, triad fractions and copolymer composition. The error vector  $\varepsilon_{\mathrm{i}}$  is often assumed to be normally distributed with a mean vector of 0, and a vxv covariance matrix  $\underline{{\mathrm{V}}}.$ , which is non- singular and may be known or unknown Since the errors of the measurements used in this work are assumed to be multi- plicative in nature, a logarithmic transfor- mation is applied resulting in

 




$$
\ln(\underline{{x}}_{i})=\ln(\underline{{\xi}}_{i})+\varepsilon_{i}
$$


(4) Thesecondstatementspeciﬁesthe model, which relates the true values of the parameters and variables,




$$
\underline{{\mathrm{f}}}(\underline{{\xi}}_{\underline{{\mathrm{i}}}},\underline{{\theta}}^{*})=0\quad\mathrm{where}\quad\mathrm{i}=1,2.\ .\ .\ .,\mathsf{n}
$$


The vector of functions f in Equation (4) is mxl, and may be linear or nonlinear in the elements of and $\xi_{\mathrm{i}}$  u. The vector contains $\underline{{\theta}}$  the reactivity ratios to be estimated. Here the functions are the triad fraction and composition models speciﬁed in Equations (1) and (2). Using a Bayesian approach, that a point estimate  $\underline{{\hat{\theta}}},$ , can be found by minimizing





(6) where  $r_{i}$  is the number of replicates at the $\mathrm{i}^{\mathrm{th}}$  trial.



The optimization algorithm uses a New- ton’s method similar to Fisher’s method of scoring to minimize  $\Phi$ . The routine to solve the EVM problem uses a nested- iterative scheme with an inner iteration that estimates the true values of the variables,, while keeping the parameter values con- stant, and an outer iteration that estimates the true values of the parameters,, while keeping the variables,, constant. The algorithm starts by using the initial para- meter estimates (supplied by the user) and the  $\boldsymbol{\upxi}^{0}$ set equal to the measured variables $\mathrm{\underline{{X}}_{i}}$ . Theinnerschemeusesthefollowing equation iteratively to update the estimates of the variables,


$$
\begin{array}{c}{{\displaystyle{\underline{{\xi}}_{i}}^{(k+1)}=\overline{{\underline{{x}}}}_{i}-{\underline{{V B^{\prime}}}}_{i}\,\left({\underline{{B}}}_{i}\,{\underline{{V B^{\prime}}}}_{i}\right)^{-1}}}\\ {{\displaystyle\qquad\times\left[f\big(\underline{{\xi}}^{(k)},\underline{{\theta}}\big)\right)+{\underline{{B}}}_{i}\big({\underline{{x}}}_{i}-\underline{{\xi}}^{(k)}\big)\big]}}\end{array
$$


(7) 
$$
\phi=\,\frac{1}{2}\sum_{i=1}^{n}r_{i}\,\left(\underline{{{x}}}_{i}-\ \underline{{{\hat{\xi}}}}_{i}\,\right)^{T}\,\underline{{{V}}}^{-1}\!\left(\underline{{{x}}}_{i}-\ \underline{{{\hat{\xi}}}}_{i}\right)
$$


where  $\underline{{\mathbf{B}}}_{\mathrm{i}}$  is the vxl vector of partial derivatives of the function with respect to the variables,


$$
B_{i}=\left[\frac{d f\,\left(\underline{{\xi}}_{i}\,,\underline{{\theta}}\right)}{d\,\left(\underline{{\xi}}_{i}\right)_{t}}\right]_{\underline{{\xi}}_{i}=\ \underline{{\xi}}_{i}(k)}
$$


(8)

### for the  $t^{t h}$ element

and $\underline{{\mathrm{V}}}$  is the vxv error covariance matrix for the measurements. Note that V must either besupplied bythe useror must be estimated from sufﬁcient replicates.

The outer scheme uses the following equation iteratively to update the estimates of the parameters,


$$
\underline{{\theta}}^{(\mathsf{u+1})}=\underline{{\theta}}^{(\mathsf{u})}-\underline{{\mathbf{G}}}^{-1}\underline{{\mathbf{q}}}
$$


(9) where  $\underline{{\mathrm{G}}}$  is the pxp expected information matrix given by,




$$
\begin{array}{r l}{G=E\left[{\frac{d^{2}\phi}{d\theta_{i}d\theta_{j}}}\right]\;}&{{}i,j\;e l e m e n t}\\ {=}&{{}\displaystyle\sum_{i=1}^{n}\;r_{i}\;\underline{{Z}}_{i}^{\prime}\;(\underline{{B}}_{i}\;\underline{{V}}B_{i}^{\prime})^{-1}\underline{{Z}}_{i}}\end{array}
$$


(10) and  $\mathtt{q}$  is the pxl gradient vector





and $Z_{\mathrm{i}}$  is the lxp vector of partial derivatives with respect to the parameters,



(12) The only assumption required for EVM is that of model adequacy, while the construction ofthe conﬁdenceregions requires that the errors are independent and identically normally distributed. Gen- erally, to quantify the uncertainty in the parameter estimates the  $95\%$  joint prob- ability contour is determined via either of the following two methods. The ﬁrst is a relatively rough approximation to the joint posterior probability region for the esti- mated parameters, and is given by




$$
(\underline{{\theta}}-\underline{{\hat{\theta}}})^{\prime}\;G(\underline{{\theta}}-\underline{{\hat{\theta}}})\;\leq\;\chi_{(p,1-\alpha)}^{2}
$$


(13) where the ‘‘ $\cdot\wedge\cdot$  indicates estimates of the parameters, G is the expected information matrix of the parameter estimates, and  $\chi^{2}$   $(\mathrm{p},\ \ 1-\upalpha)$  l  a) is the value of the chi-squared distribution having  $\mathfrak{p}$  degrees of freedom, exceeded with probability  $\alpha.^{[1]}$  Here the assumption of multivariate normality is made and the elliptical posterior probabil- ity contour is determined. It is important to note that both the shape and probability content of this contour are approximate.

 



The second method is a somewhat better alternativetoapproximatingthejoint posterior probability region (JPR), and is given by the equation


$$
\phi\big(\underline{{{\theta}}}\big)=\phi\big(\underline{{{\hat{\theta}}}}\big)+\frac{1}{2}\,\chi_{(p\mathrm{~,~}1-\alpha)}^{2}
$$


(14) 
$$
\begin{array}{r l}{\underline{{q}}=\left[\frac{d\phi}{d\theta_{j}}\right]}&{{}i^{t h}\;e l e m e n t}\\ {=}&{{}\sum_{i=1}^{n}r_{i}\;\underline{{Z^{\prime}}}_{i}(\underline{{B}}_{i}\;\underline{{V B^{\prime}}}_{i})^{-1}\underline{{B}}_{i}(\underline{{\overline{{x}}}}_{i}-\underline{{\hat{\xi}}}_{i})}\end{array}
$$


where  $\phi$  is given by Equation (6). This approximation gives contours of the correct 
$$
\underline{{\underline{{Z_{i}}}}}=\left[\frac{d f(\underline{{\xi}}_{i},\:\theta)}{d\theta_{k}}\right]\qquad\quad k^{t h}\;e l e m e n t
$$


shape (not a general elliptical shape) but of approximateprobabilitycontent.All contours shown in this paper are calculated using equation (14).

## Results and Discussion

The following case studies were analyzed in order to: (1) comparereactivity ratio estimates obtained from composition data with those calculated from triad fraction data,(2)determineiftriadfractions resulted in less uncertainty in the parameter estimates; and (3) evaluate the potential improvement in the estimates when the triad fractions and composition are com- bined.

Copolymer Systems Case I: Acrylamide/Acrylonitrile Mukherjee et al. studied the acryla- minde/acrylonitrile (AN) system at $65\,^{\circ}\mathrm{C}$  in bulkcopolymerizationandpresented experimental data for both copolymer composition and triad fractions. The JPR’s for the different data types can be seen in Figure 1. The JPR based on composition data is much larger than the JPR obtained when using triad fraction data. Further- more, the JPR obtained from triad fraction analysis, falls within the JPR obtained from analysis of the composition data. For this system composition data and triad fraction data lead to consistent reactivity ratio estimates and triad fraction data leads to estimates with a smaller amount of uncer- tainty as quantiﬁed by the areas of the JPR. Diagnostic checking was performed in order to determine whether the reactivity ratio estimates in fact model the experi- mental results accurately. The reactivity ratio estimates, obtained from analyzing the triad fractions and composition data separately, were used in the triad fraction equations (Eq. 1) to compare predicted triad fractions with the published experi- mental triad fractions. It can be seen in Figures 2 a) – d) that the predicted triad fractions,calculatedusingeitherthe composition-based or the triad fraction- based reactivity ratio estimates, are in good agreement with the experimental triad fractions results. Furthermore, the copoly- mer composition can be predicted well with reactivity ratios that are estimated from either composition data or triad fraction data, as seen in Figure  $2\mathrm{~e~}$ ). These results conﬁrm that the estimates obtained from either data type are consistent.

  

 



Figure 3 shows the JPR obtained by combining triad fraction with composition dataandcomparesitwiththeJPR calculated using only triad fractions. This plot was produced to determine if combin- ing triad fractions with composition would produce even better parameter estimates. It can be seen in Figure 3 that the use of composition data in conjunction with the triad fractions data does not result in a signiﬁcant decrease in uncertainty, com- pared to simply using triad fraction data. Subsequently, for this system, on might argue that copolymer composition data does not contribute signiﬁcant additional information to the estimation of reactivity ratios, given that triad fraction data is readily available.

 



Case II: Acrylonitrile/Acrylic Acid Brar and Dutta studied the acrylonitrile (AN)/acrylic acid (AA) system at  $27^{\circ}$  in bulk photo-initiated copolymerization and presented experimental data on both copo- lymer composition and triad fractions. The data are shown in Table 2.

Note that a total of 10 experiments are available in which copolymer composition data was measured, eight in which triad fractions were measured and only six for which both composition and triad fractions are available. The different number of data points and the difference in the feed compositions at which data are available complicate the case study somewhat. In the article the composition data was analyzed to obtain reactivity ratio point estimates, which were then used to obtain predicted values for the triad fractions. Model predic- tions of triad fractions based on composition- based reactivity ratio estimates were found to be in good agreement with the experi- mentally determined triad fractions. Here the consistency between the copolymer composition and triad fractions is explored by comparing the resulting reactivity ratio estimates for the two different measure- ments. From Figure 4 it can be seen that the JPR of the triad fraction analysis is smaller in area than the composition JPR showing again that the use of triad fraction data results in more precise reactivity ratio estimates, compared with those obtained from composition data. In comparison to the previous example the two JPRs do not overlapasmuch.Diagnosticchecking performed for the AN/AA system showed exactlythesameresultsaswiththe diagnostic checking for the previous case study. In other words, when the composition- based and triad fraction-based estimates were used to predict the experimental results good agreement was again found. Therefore for this system, the estimates are again consistent.



Figure 5 shows the comparison between the results obtained when combining triad fractions with composition and just using triad fractions. Again the regions are of approximately the same size, indicating no signiﬁcant difference in parameter preci- sion. Note that the combined data contour lies above that for triad fraction data only. The latter in turn lies above that based on composition data (Figure 4). Normally one would expect the combined contour to fall somewhere between the composition- and triad fraction-based contours. The explana- tion here is related to the differences in feed compositions between the different data sets.

Case Study III: Acrylonitrile/Pentyl Methacrylate Brar and Dutta also studied the acryloni- trile (AN)/ pentyl methacrylate (PMA) system at room temperature in bulk and presented experimental data on both copo- lymer composition and triad fractions.



Figure 6 shows the JPR from the EVM analysis of the composition data to be much larger than the JPR from the EVM analysis of the triad fraction data. That is, this case study is further testimony to the fact that the use of triad fraction data results in less uncertainty in the reactivity ratio para- meter estimates. In this case, however, there is a considerable difference in the estimates obtained, in that the composition- based estimates are considerably larger.

 



Similar to the procedure in the other case studies, diagnostic checking was then performed to ensure the ﬁt of the model to the experiment results. Figures  $^{7\mathrm{a-d}}$  show the predicted triad fractions obtained when using the composition-based and triad fraction-based reactivity ratio estimates, while Figure 7 e shows the copolymer com- position predicted from the Mayo-Lewis equation. In this case, triad fractions are best predicted using triad-based reactivity ratio estimates. Composition-based esti- mates do not yield very good triad fraction predictions and triad fraction-based esti- mates do not yield very good copolymer composition predictions. In addition, triad fraction A121 is not predicted well by either set of reactivity ratios. There is an observed lack of ﬁt present. One possible explanation might be related to the calculation required to convert the actual measurements, namely the NMR normalized peak areas, into triad fractions. These calculations require addi- tional parameters whose values might be inconsistent for this triad fraction. The peak areas and the converting equations are unfortunately not reported. Hence the source for the lack of ﬁt is unknown.



Finally, combining copolymer composi- tion and triad fraction data showed the same trends as seen in previous examples.

Case Study IV: Styrene/Acrylonitrile A number of groups have published work on the styrene (STY) – acrylonitrile (AN) system for both bulk and solution copoly- merization. Here we analyzed data from the groups of Hill et al. and Klumper- man and Kraeger. Data is reported for both bulk and in toluene solvent.

A clearer picture of the estimates from the different data types can be seen when the data is grouped together by system (i.e., bulk and toluene solvent) and the JPR’s are shown. Note that in the ﬁgures of this case study, if the data set used in the analysis is taken from Hill et al, Hill et al. or Klumperman and Kraeger, it is referred to in the ﬁgure as (Hill, 1989), (Hill, 1992) or(Klumperman,1994),respectively. Furthermore, in all cases monomer 1 and 2 refer to styrene and acrylonitrile mono- mer, respectively.



In Figure 8 the results are shown for the Hill et al. triad fractions and composi- tion data sets for bulk copolymerizations. This plot indicates relatively large differ- ences between composition-based results and triad fraction-based estimates similar to the results observed for the AN/PMA system. The two different triad fraction data sets yield results which are relatively close. Furthermore, by considering Figure 9 with the bulk STY/AN studies from Hill et al. and Klumperman and Kraeger, it can be seen that the regions from the different data sources do not overlap, for neither composition nor triad fraction data.

 



Similar trends are again seen when consideringtheSTY/ANstudiesin toluene (see Figure 10). Note however that the joint probability regions are closer together than they are in the analysis of the bulk experiments.



A possible explanation for the trends observed over the four different copolymer systems may have to do with the varying degrees of heterogeneity (preferential sorp- tion or solvent effects) exhibited due to phase separation or precipitation during copolymerization. These ‘‘heterogeneity’’ effects became progressively more signiﬁ- cant as we moved from case study I to IV. The effects have been documented before by several researchers for several copoly- mer systems involving AN and carboxylic acids. An increasing degree of hetero- geneity leads to increased difﬁculty in obtaining polymer samples that give repre- sentative triad fraction measurements that are consistent with composition data. This may explain why reactivity ratio estimates from triad fractions on the one hand and composition on the other compare favour- ably for the acrylamide/AN and the AN/ AA systems, but do not for the AN/PMA and STY/AN systems. Further support for this hypothesis is seen in Figure 10, where the results are more consistent when STY/ AN is copolymerized in toluene solvent, which will make the system less hetero- geneous.



## Concluding Remarks

Thispaperexaminedthequestionof whether or not triad fraction data can lead NOTE:  $\mathsf{\Delta}=\mathsf{Y e s},\,\mathsf{X}\!=\!\mathsf{N o}.$  $\surd=$  The following abbreviations are used: Klump $=$ Klumperman and Kraeger; Hill ¼ Hill et al.. Blank spaces in the table indicate that the analysis was not performed for that particular entry.



to reliable reactivity ratio estimates. Sev- eral aspects were investigated, namely: (1) Are triad fraction-based estimates consis- tent with composition-based estimates, (2) Do triad fractions lead to reactivity ratio estimates with less uncertainty, and (3) Is there and advantage to combining triad fraction and copolymer composition data. These questions were explored using four different case studies involving low con- version data from four different copolymer systems. Table 3 summarizes the results of these investigations.

In general triad fraction data did con- sistently result reactivity ratio estimates havingsigniﬁcantlysmalleruncertainty compared to those obtained from composi- tion data. Combining triad fractions with copolymer composition did not reduce the uncertainty in the reactivity ratio estimates (therefore leading to higher quality esti- mates), compared to those obtained only from triad fractions. The reactivity ratio estimates may not agree for the same system, when different data types are used. That is, the parameter estimates obtained from analyzing triad fraction data are not always in agreement with the estimates obtained from analyzing composition data. Heterogeneityinthecopolymerization process may provide an explanation for this observation.

Acknowledgements: Financial support from the Natural Science and Engineering Research Council of Canada (NSERC), the Canada Research Chair (CRC) program, and a Province of Ontario Graduate Scholarship (OGS) is gratefully acknowledged.

[1] A. L. Polic, T. Duever, A. Penlidis, J. Poly. Sci. Polym. Chem. 1994, 36, 813.

[2] P. Rossignoli, T. Duever, Polym. React. Eng. 1995, 3, 361.

[3] M. Dube, R. Sanayei, A. Penlidis, K. O’Driscoll, P. Reilly, J. Polym. Sci., Polym. Chem. 1991, 29, 703.

[4] A. Brar, K. Dutta, J. Appl. Polym. Sci. 1998, 69, 2507. [5] A. Brar, K. Dutta, Eur. Polym. J. 1998, 34, 1585.

[6] A. Brar, B. Jayaram, K. Dutta, J. Polym. Mater. 1994, 11, 171.

[7] A. Brar, M. Kaur, J. Appl. Polym. Sci. 2003, 88, 3005. [8] A. Brar, M. Kaur, Eur. Polym. J. 2003, 39, 705.

[9] A. Brar, D. Pradhan, J. Appl. Polym. Sci. 2003, 89, 1779.

[10] A. Brar, D. Pradhan, J. Mol. Struct. 2003, 649, 245. [11] A. Brar, R. Kumar, M. Kaur, J. Mol. Struct. 2003, 650, 85.

[12] P. Brown, K. Fujimori, Polymer 1994, 35, 2302. [13] A. Burke, T. Duever, A. Penlidis, Macromol. Theory Simul. 1994, 3, 1005.

[14] J. de la Fuente, M. Fernandez-Garcia, M. Fernandez- Sanz, E. Madruga, Macromolecules 2001, 34, 5833. [15] I. Erol, F. Yavuz, M. Durgun, Polym. J. 2004, 36, 303.

[16] F. Escher, G. Galland, J. Polym. Sci., Polym. Chem. 2004, 42, 2474.

[17] A. Ferando, A. Longo, Polym. Prepr. 1997, 38, 798. [18] C. Fernandez-Monreal, M. Sanchez-Chaves, G. Martinez, E. Madruga, Acta Polym. 1999, 50, 408.

[19] M. Galimberti, F. Piemontesi, O. Fusco, I. Camurati, M. Destro, Macromolecules 1998, 31, 3409.

[20] N. Ha, Comp. Theor. Polym. Sci. 1998, 7, 121.

[21] T. Ha, K. Fujimori, D. Tucker, P. Henry, J. Macromol. Sci., Pure Appl. Chem. 1999, A36, 893.

[22] D. Hill, J. O’Donnell, P. O’Sullivan, Eur. Polym. J. 1989, 25, 911.

[23] D. Hill, A. Lang, P. Munro, J. O’Donnell,Eur. Polym. J. 1992, 28, 391.

[24] S. Hooda, A. Brar, J. Appl. Polym. Sci. 2003, 88, 3232.

[25] Y. Kim, H. Harwood, Polymer 2002, 43, 3229.

[26] Y. Kim, D. Priddy, H. Harwood, Polymer 2003, 44, 4165.

[27] B. Klumperman, I. Kraeger, Macromolecules 1994, 27, 1529.

[28] I. Kramer, H. Pasch, H. Handel, K. Albert, Macro- mol. Chem. Phys. 1999, 200, 1734.

[29] I. Maxwell, A. Aerdts, A. German, Macromolecules 1993, 26, 1956.

[30] M. Mukerjee, S. Chatterjee, A. Brar, J. Appl. Polym. Sci. 1999, 73, 55.

[31] K. O’Leary, D. Paul, Polymer, 2004, 45, 6575.

[32] P.Pazhanisamy,M.Ariff,Q.Anwaruddin, J. Macromol. Sci., Pure Appl. Chem. 1997, A34, 1045. [33] A. Rudin, K. O’Driscoll, M. Rumack, Polymer 1981, 22, 740.

[34] C. Selvamalar, P. Vijayanand, A. Penlidis, S. Nanjundan, J. Appl. Polym. Sci. 2004, 91, 3604.

[35] C. Soyhan, I. Erol, J. Appl. Polym. Sci. 2004, 91, 964. [36] S. Sundarrajan, K. Ganesh, K. Srinivasan, Polymer, 2003, 44, 61.

[37] S. Wharry, Polymer 2004, 45, 2985.

[38] Y. Zhao, P. Liu, H. Liu, J. Jiang, C. Chen, F. Xi, Macromol Rapid. Commun. 2001, 22, 633.

[39] J. Koenig, ‘‘Chemical microstructure of polymer chains’’, John Wiley & Sons Ltd, New York 1980.

[40] P. Reilly, H. Patino-Leal, Technometrics 1981, 23, 221.

[41] H. Patino-Leal, P. Reilly, K. O’Driscoll, J. Polym. Sci. Polym. Lett. 1980, 18, 219.

Macromol. Symp. 2008, 271, 48–6363

[42] S. Keeler, P. Reilly, Can. J. Chem. Eng. 1991, 69, 27. [43] P. Reilly, S. Keeler, H. Reilly, Appl. Stat. 1993, 42, 693.

[44] L. Garcia-Rubio, A. Hamielec, J. Appl. Polym. Sci. 1979, 23, 1397.

[45] L.Garcia-Rubio,G.Lord,J.Macgregor,A. Hamielec, Polymer 1985, 26, 2001.

[46] K. Plochocka, H. Harwood, Polym. Prepr. 1978, 19, 240.

[47] Y. Semchikov, Macromol. Symp. 1996, 111, 317.
# Comparison of the Immunity Status in-Between Children with  $\upbeta\cdot$ -Thalassaemia Major Receiving Different Treatment Modalities: A Single Egyptian District Study

Mahmoud SS, Mohamed GB, Hakeem GLA, Higazi AM2, Nafady AAH, Farag NM2, Mahrous DM1, Kamal NN4, Hassan Z1, Aziz EA1 and Shaban A1 Department of Pediatrics, Faculty of Medicine, Minia University, Minia, Egypt Department of Clinical Pathology, Faculty of Medicine, Minia University, Minia, Egypt Department of Clinical Pathology, Faculty of Medicine, Assuit University, Assuit, Egypt Department of Public Health, Faculty of Medicine, Minia University, Minia, Egypt

   

  

 



\*Corresponding author: Gehan Lotfy Abdel Hakeem, Department of Pediatrics, Faculty of Medicine, Minia University, Minia, Egypt, Tel: +201001818254; E-mail: gehanlotfy72@yahoo.com Received date: November 29, 2016; Accepted date: January 08, 2017; Published date: January 16, 2017 Copyright:  $\copyright2017$ Mahmoud SS, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

 



### Abstract

Background:  $\upbeta$ -thalassaemia major is one of the chronic hemolytic anemias resulting from defect in  $\upbeta\cdot$ -globin chain. It requires frequent blood transfusion plus other treatment modalities. These treatment modalities may be associated with certain immunologic modulations.

Objective: To assess the immunity status in children with  $\upbeta$ -thalassaemia major under different treatment regimens within El Minia, Egypt.

Subjects and Methods: One hundred forty-four children were enrolled and classified into four groups. Thirty-six  $\upbeta$ -thalassaemia patients treated only with blood transfusion (group I). Thirty-six patients treated with transfusion and iron chelation (group II). Thirty-six patients treated with transfusion, iron chelation and subjected to splenectomy (group III). Group IV involved thirty-six apparently healthy age and sex matched children. CBC plus serum levels of ferritin, IgA, complement C3 and C4 were measured along with detection of  $^{\mathrm{CD3+}}$ ,  $C D4+$ ,  $\mathsf{C D8+}$ ,  $\mathsf{C D}\,19+$ and  $\mathsf{C D56+}$  lymphocyte percentages and absolute counts.

Results: IgA levels were significantly higher in thalassaemia patients compared to controls  $(\mathsf{p}\!\!<\!0.001)$  plus highly significant increase in IgA levels in splenectomized patients than non-splenectomized  $(\mathsf{p}\!\!<\!0.001)$ . Levels of C3 were significantly decreased in all patients compared with controls  $(\mathsf{p}\mathrm{=}0.001)$  with a highly significant decrease in C3 levels in splenectomized patients than non splenectomized ones  $(\mathsf{p}\!\!<\!0.001)$  but no statistical difference between their C4 levels. Significant statistical differences were revealed regarding  ${\mathsf{C D3+}}$ ,  $C D4+$ and  $^{\mathrm{CD8+}}$ T lymphocyte percentages within thalassaemia groups when compared to each other’s and to controls. Splenectomized patients had higher significant levels regarding serum ferritin  $(\mathsf{p}\mathsf{=}0.02)$  along with  $^{\mathrm{CD3+}}$ ( $\mathsf{p}\mathrm{=}0.05)$ ,  $\mathsf{C D4}+(\mathsf{p}\mathsf{=}0.05)$ and  $\mathsf{C D8+}$   $(\mathsf{p}\mathrm{=}0.037)$  lymphocyte percentages compared to non-splenectomized.  $\mathsf{C D}\,19+$ lymphocyte percentages were significantly higher while  $\mathsf{C D56+}$ lymphocyte percentages were significantly lower in all patients compared with controls  $(\mathsf{p}\!\!=\!\!0.02$  and 0.05).

Keywords: Thalassaemia; Immunity; Treatment modalities; Egyptian children; El Minia region Abbreviations: CBC-Complete Blood Count; IgA-Immunoglobulin A



## Introduction

Thalassaemia is defined as a group of inherited disorders that arise as a result of certain mutations in hemoglobin (Hb) genes.  $\upbeta\cdot$ - thalassaemia major (βTM) has a high prevalence in the Mediterranean region, including Egypt, Middle East, Indian subcontinent, and South East Asia [1]. Though, it is a growing global health problem due to extensive population migrations. About  $1.5\%$  of the world populations are carriers of  $\upbeta$ -thalassaemia gene [2]. βTM patients present with many problems rather than severe anemia, including increased susceptibility to infections which constitutes the second most common cause of mortality and a major cause of morbidity in  $\upbeta$ -thalassaemia after heart failure [3]. The underlying causes of their increased liability to infections may be attributed to anemia per-se, reticuloendothelial system dysfunction as well as therapeutic regimens-related infections [4].



These therapeutic approaches incorporate repeated blood transfusion alone or in combination with iron chelators. Splenectomy is added when indicated. Recently, gene therapy and bone marrow transplant or stem cell therapy are promising hopes [5]. Volumetric and multiple transfusions lead to alloimmunization, accumulation of iron in different tissues and are associated with increased risk of infections. These infections are caused by lots of bacterial and viral infections such as cytomegalo virus, Ebstein-Barr virus and hepatitis C virus [6]. Likewise, Yersinia enterocolitica is usually associated with the use of deferoxamine (DFO) as an iron chelator [7]. As well, the risk of sepsis in splenectomized patients is as high as  $7\%$ over a 10-years period and almost  $25\%$  of splenectomized patients are at the risk of severe infections [8]. Hence, it is crucial to identify the impact of these treatment modalities on the immune status of these patients.



Several studies about the immune competence in  $\upbeta$ -thalassaemia had revealed numerous quantitative and functional defects, involving T and B lymphocytes, immunoglobulins in addition to the impairment in components of the complement system [9,10]. Additionally, immunologic disorders in patients with  $\upbeta\mathrm{TM}$ comprise decreased absorption and phagocytic ability of segmented neutrophils, changes in cytokines production as well as dysfunction of macrophages, properdin and lysozymes [11]. As well, it has been reported that the distribution of  $\upbeta$ -globin gene mutations differs among ethnic groups plus its regional and individual variation [12]. Furthermore, there is increasing evidence suggesting that clinical features or complications of  $\upbeta\mathrm{TM}$  plus their immune characters and even their response to various therapies are population dependence. Also, the immunity status of  $\upbeta\mathrm{TM}$  patients varies in relation to the type and extent of treatment approaches [8].

The overall objective of this study was to identify the immunological alterations of  $\upbeta\mathrm{TM}$  patients within Egyptian children in El Minia district. Our aim was to compare the immune competence between those  $\upbeta\mathrm{TM}$  children who receiving multiple blood transfusions alone or with iron chelating agents plus or minus splenectomy. We studied serum immunoglobulin A (IgA) levels beside serum C3 and C4 complements in addition to blood counts of T, B and natural killer (NK) lymphocytes.

## Subjects and Methods

### Subjects

This multicenter, case-control study was conducted in El Minia city in Egypt. Patients were selected from Pediatric Hematology Outpatient Clinic at Minia University Hospital and from Pediatric Hematology Outpatient Clinic, National Blood Bank, Minia Ministry of Health while controls were selected from apparently healthy children from Pediatric Growth Clinic, Minia University Hospital. This study was carried out between May and December, 2015. One hundred and forty-four children were included in this study, which involved a group of 108 thalassaemia patients and 36 apparently healthy children with matching age and sex as a control group IV. Control children were 18 males  $(50\%)$  and 18 females  $(50\%)$ . There ages were ranged from 2-16 years old. The thalassaemia patients group was further subdivided into three subgroups according to the difference in their treatment regimens. These subgroups comprised patient groups from I-III. Group I patients (No. 36) were receiving blood transfusion only. These patients were 18 males  $(50\%)$  and 18 females  $(50\%)$ . There ages were ranged from 2-8 years old. Patients in group II (No. 36) were receiving blood transfusion and iron chelation while patients in group III (No. 36) were receiving blood transfusion plus iron chelation and also subjected to splenectomy. Group II patients were 12 males  $(33.3\%)$  and 24 females  $(66.7\%)$ . There ages were ranged from 5-12 years old. Group III patients were 21 males  $(58.3\%)$  and 15 females  $(41.7\%)$ . There ages were ranged from 10-16 years old.



All included patients and controls were subjected to complete history taking (name, age, sex, residence, family history, consanguinity, age of 1st blood transfusion, amount of transfusion per year, duration of splenectomy in splenectomized patients in addition to history of exposure to infections). Moreover, the involved children were examined carefully (general examination, anthropometric measures which were plotted on percentile growth charts, vital data as well as examination of chest, heart and abdomen). The exclusion criteria were patients who had a bone marrow transplant. Finally, the laboratory investigations were performed to all subjects. These investigations were comprised Complete Blood Count (CBC), blood smears, Hemoglobin  $\mathrm{(Hb)}$  electrophoresis for controls, serum ferritin, serum level of immunoglobulin A (IgA), complements C3 and C4 plus percentages and absolute counts of CD3, 4, 8, 19 and 56 T lymphocytes.

### Blood Sampling

Venous blood samples were collected from both patients and controls under complete aseptic conditions. About  $2\;\mathrm{ml}$ of blood were withdrawn in K3-EDTA anticoagulant tubes for complete blood counts, peripheral blood smears as well as flow cytometric analysis (immunophenotyping). About 3ml of blood were withdrawn in a plain tube without any anticoagulant. Their serum was then separated and stored at -  $70^{\circ}\mathrm{C}$ till used for serum ferritin, IgA and complements assays.

Blood samples were withdrawn from thalassaemia patients just prior to a scheduled transfusion. The blood samples from the control group were taken while coming for follow-up of their growth. All controls had normal Hb levels for their age and sex with normal red blood cell indices. At the time of sampling all patients and controls were apparently free from infections (normal C-reactive protein).

### Laboratory methodology

Complete blood count was performed using an automated blood counter (Sysmex KX-21N). Additionally, peripheral blood smears were stained by Leishman stain. Reticulocyte counts were detected as well. The absolute lymphocytes and neutrophils counts were calculated. Serum ferritin levels were measured using human in vitro Enzyme- Linked Immunosorbent Assay (EIA) kit from Abcam Inc, Cambridge, USA. The assay was performed according to the manufacturer’s instructions. Ferritin values were expressed as nanogram per milliliter  $\left(\mathrm{ng/ml}\right)$ . [13,14] Levels of IgA were determined using human in vitro EIA kit from Abcam Inc, Cambridge, USA according to the manufacturer’s instructions. Final IgA values were expressed in milligram per deciliter  $(\mathrm{mg/dl})$ . As well, Complements C3 and C4 serum levels were detected with human in vitro EIA kit from Abcam Inc, Cambridge, USA. Complements C3 and C4 values were expressed in milligram per deciliter  $(\mathrm{mg/dl})$ .



### Flow cytometric immunophenotypic analysis

T- Lymphocyte subsets in whole blood samples were enumerated using fluoroisothiocyanate (FITC) conjugated CD4 (Becton Dickinson, Bioscience, USA), phycoerythrin (PE) conjugated CD8 (Becton Dickinson, Bioscience, USA) and peridinium-chlorophyll- protein (Per-CP) conjugated CD3 (Becton Dickinson, Bioscience, USA). B-Lymphocyte subsets in whole blood samples were enumerated using phycoerythrin (PE) conjugated CD19 (Becton Dickinson, Bioscience, USA). Natural killer Lymphocyte subsets in whole blood samples were enumerated using phycoerythrin (PE) conjugated CD56 (Becton Dickinson, Bioscience, USA). Flow cytometric analysis was done by FACS Calibur flow cytometry with Cell Quest software (Becton Dickinson Biosciences, USA). An isotype matched negative control was used with each sample. Forward and side scatter histogram was used to define the lymphocyte population (R1). The absolute counts and percentages of CD3, 4, 8, 19 and 56 subsets of lymphocytes were calculated.

 



### Statistical analysis

The clinical and laboratory data were recorded. These data were analyzed using statistical program for social science SPSS software version 17 (SPSS Inc., Chicago, IL, USA). Quantitative variables were presented as mean  $\pm$ standard deviation (SD). On the other hand, description of qualitative variables was presented as number (No.) and percentage  $(\%)$ . Results were expressed as tables and figures. Graphics were done by Excel, Microsoft Office 2010. Student t-test was used to compare results between groups as regards quantitative variables. The correlation was performed by using Pearson correlation coefficient (r). For all tests, a probability (p) was considered non-significant if  ${>}0.05.$ , significant if  $\leq0.05$  and highly significant if  $\leq0.001$ . One Way ANOVA test was used for comparison of quantitative data between more than two groups. The Chi-square test was used to compare qualitative variables between groups.



## Results

### Comparison of laboratory results and immunological parameters between  $\upbeta\mathrm{.}$ -thalassaemia major children under different treatment modalities and controls

A significant difference was found on comparison between all thalassaemia patients (group I, II, III) and control group regarding Hb levels  $\scriptstyle(\mathrm{p}=0.015)$ , ANC  $\mathrm{(p=0.05)}$ , serum IgA  $(\mathrm{p}{<}0.001)$ , serum C3  $\scriptstyle(\mathrm{p}=0.001)$ , serum C4 ( $\mathrm{\Deltap=}0.002\mathrm{\rangle}$ ,  $\mathrm{CD}3+$  absolute count  $\mathrm{(p=0.05)}$ ,  $\mathrm{CD3+/}$   $\mathrm{CD4+}$  absolute count  $\mathrm{(p=0.04)}$ ,  $\mathrm{CD3+/CD8+}$  absolute count  $\mathrm{(p=0.06)}$ ,  $\mathrm{CD19+}$  absolute count  $\mathrm{(p=0.02)}$  and  $\mathrm{CD56+}$ absolute count  $\mathrm{(p=0.05)}$  (Table 1).

Also, differences within the 3 thalassaemia groups regarding IgA, C3, C4 (Table 2) along with percents of  $\mathrm{CD}3+$ ,  $\mathrm{CD3+/CD4+},$ ,  $\mathrm{CD}3+/$   $\mathrm{CD8+},$ ,  $\mathrm{CD19+}$  and  $\mathrm{CD56+}$  as well as their absolute counts were summarized (Figure 1 and Table 2).

Table 2: Comparison between studied patients regarding immunological markers; Group I: thalassaemia children receiving blood transfusion only; Group II: thalassaemia children receiving blood transfusion  $^+$ iron chelation; Group III: thalassaemia children receiving blood transfusion  $^+$ iron chelation  $^+$ splenectomy; vs: versus; Ig A: Immunoglobulin A; C3: Complement C3; C4: Complement C4; Non sig.  ${>}0.05.$ , Significant $\leq0.05^{\star};$ , High Significant  $\leq0.001^{\ast\ast}$ .

non splenectomized groups  $(\mathrm{p}{<}0.001)$  (Figure 2A). Moreover, splenectomized patients showed significant decrease in serum C3 levels  $_{(\mathrm{p<0.001})}$  and non-significant difference in serum C4 levels in comparison to non splenectomized groups  $\scriptstyle(\mathrm{p}=0.282)$  (Figure 2B).

 $\mathrm{CD}3+,$   $^{4+}$  and  $^{8+}$ percentages were statistically significant higher in the splenectomized group in comparison to non splenectomized patients  $\scriptstyle(\mathrm{p}=0.05,\ 0.05$ and 0.037 respectively). On the other hand, splenectomized children showed non-significant difference from non splenectomized ones concerning  $\mathrm{CD19+}$ percentage, but highly statistically significant lower levels regarding  $\mathrm{CD56+}$ ( $\mathrm{{p=}}0.235$ , <0.001 respectively) (Figure 2C).

### Serum ferritin levels within  $\upbeta.$ -thalassaemia subgroups in correlation to immunological characteristics

The mean  $\pm\ \mathrm{SD}$ of serum ferritin levels within thalassaemia groups as well as controls were shown in table 1. There was a significant increase in serum ferritin levels when patients were compared with controls  $\scriptstyle(\mathrm{p}=0.013)$ . Additionally, serum ferritin levels were elevated in splenectomized patients (group III) compared with non splenectomized ones  $\mathrm{(p=0.02)}$  (Figure 3). Significant moderate positive correlation was found between  $\mathrm{CD}3+$ cells and ferritin  $\mathrm{(p=0.04,r=0.6)}$ . As well, significant fair positive correlation was found between  $\mathrm{CD3+/}$   $\mathrm{CD8+}$  cells and ferritin  $\mathrm{(p=0.04,r=}0.3\mathrm{)}$  in group I. There were no other significant correlations between ferritin and the remaining immune parameters (Table 3).

The comparison between splenectomized patients (group III) and non splenectomized ones (group I plus II) concerning the studied immune parameters was illustrated in Figure 1. Serum IgA levels were statistically significant high in splenectomized patients compared with

## Discussion

It had been shown that the immune-capability of  $\upbeta\mathrm{TM}$ patients has been altered [9, 10]. The presence of abnormal erythrocytes leads to continuous activation of monocytes and immune clearance [15]. Furthermore, it has been shown that  $\upbeta\mathrm{TM}$ -related immunologic changes differ according to the variety and duration of therapeutic procedures and in a population dependence manner [8]. Data relevant to the impact of different treatment modalities on the immune system in thalassaemia patients are yet limited as these data should be adjusted for geographical differences. The aim of this work was to evaluate the immune status of Egyptian children within El Minia region who suffering from  $\upbeta\mathrm{TM}$ and receiving various treatment modalities.

Immunoglobulins and complement proteins play a pivotal role in humoral immunity. Therefore, we assessed the deviations of these parameters among thalassaemia patients according to their treatment regimens [16]. This study has been concerned with assessment of IgA, serum complement C3 and C4 levels. IgA is an important mediator of mucosal immunity along with its involvement in complement activation via alternative and lectin pathways. Additionally, complement C3 and C4 are among the most commonly measured complement components [17].

The present study demonstrated that serum IgA levels were significantly higher in all groups of  $\upbeta\mathrm{TM}$  compared to healthy controls  $(\mathrm{p}{<}0.001)$ . This was similar to what was detected by Ali N. Salman, et al., Jeddoa et al. and Ghaffari et al. [16,18,19]. On the other hand, Ezer et al. showed that serum levels of IgA were normal within all subgroups of βTM. In Ezer study, all patients with βTM appeared to have an intact humoral immune system whatever their criteria. In addition, our data showed that splenectomized patients had higher significant increase in serum IgA levels than non splenectomized ones  $(\mathrm{p}{<}0.001)$  [20]. Many previous studies had supported our data, but Ahluwalia et al. didn’t detect any impacts of splenectomy on immunoglobulins [21-24].



The mechanisms of such increase in serum IgA levels can be attributed to iron overload in the skin of thalassaemia patients which leads to stimulation of the mucocutaneous IgA production [25]. As well, it has been suggested that iron overload causes an increased migration of T helper cells to the gut and lymph nodes which increases serum immunoglobulins levels especially IgA as the main immunoglobulin isotype in most mucosal surfaces [18]. Also, iron and its binding proteins have immune regulatory properties. Therefore, iron excess may tip the immune balance unfavorably to allow increased growth rates of infectious organisms followed by increased immunoglobulin levels [8]. Moreover, repetitive transfusions lead to continuous alloantigenic stimulation [26,27]. Additionally, thalassaemia patients are prone to many bacterial and viral infections [3]. Therefore, the immune system is activated and immunoglobulin levels are elevated [8]. Finally, increased serum levels of IgA mostly after splenectomy could be due to the pressure that occurs on other secondary lymphoid organs to synthesis major immunoglobulin classes for compensation of spleen loss [28].



In the current study, the mean levels of complements C3 and C4 were significantly decreased in all patient groups compared with controls  $\scriptstyle(\mathrm{p}=0.001\$ , 0.002 respectively). This is in accordance with Jeddoa et al., El Yazji and Amin et al. studies. [16,23,29] Ezer et al. results are unlike ours as their study showed no significant difference in C3 levels between patients and controls [20]. This may be explained on the basis that repeated blood transfusion could result in continuous exposure to various antigens which lead to continuous immune system stimulation and hence complement consumption [8]. The addition of iron chelators in our study led to significant elevation of complement C3 and C4 along with increased serum IgA levels in Group II than Group I ( $\mathrm{\Deltap=}0.01$ , 0.03 and 0.04 respectively). Aleem A et al. showed significant immune abnormalities in  $\upbeta\mathrm{TM}$ patients with oral iron chelators including low C4 level [30]. As well, we found a highly significant decrease in serum C3 levels in splenectomized patients than non splenectomized ones  $(\mathrm{p}{<}0.001)$ . This was the same as Darzi AA et al. results [21] and unlike Kiani-Amin et al. outcomes [31]. The spleen plays a role in releasing starter proteins of complement activation pathways. So, splenectomy causes reduction in complement components [21]. Also, splenectomized patients have the most increased risk of overwhelming infections which stimulate the immune system, leading to complement consumption [23,32].

Lymphocytes are crucial in cell mediated immunity. Lymphocyte subpopulations include T cells, B cells and NK cells. T lymphocytes contain  $\mathrm{CD4+}$ T cells, which become activated upon foreign antigen exposure, such as intracellular pathogens, fungi and protozoa in addition to  $\mathrm{CD8+}$ T-cells which destroy virally infected cells.  $\mathrm{CD4+}$  cells along with  $\mathrm{CD8+}$ cells represent the majority of T lymphocytes [33]. We estimated the absolute counts along with percentages of  $\mathrm{CD}3+$ lymphocytes as representative of total T Lymphocytes (pan-T cell marker) besides  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ T subsets. Since absolute counts can vary from day to day and according to child age, it is more useful to look at the percentages of these lymphocyte subpopulations [34,35]. We found significant statistical differences regarding percentages of  $\mathrm{CD}3+,$   $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ cells within thalassaemia groups when compared to each other’s and to healthy controls. Patients of group I had significantly elevated  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages, when compared to controls and to Group II patients. Group II patients had significantly lower  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages when compared to other thalassaemia groups but group III had significantly elevated  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages when compared to controls and other thalassaemia groups. These findings were in agreement with Kadam et al., Gharagozloo et al. and Vento et al. [3,25,36]. In contrast, Noulsri, et al. and Del Vecchio et al. had reported an insignificant difference in  $\mathrm{CD}3+$ cells and in its  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ subsets between patients and controls [37,38]. Blood transfusion causes chronic infections and thus constant stimulation of the immune system which might induce T cells. Besides, the majority of patients in this study were found to have increased  $\mathrm{CD4+}$  and  $\mathrm{CD8+}$ cell percentages except those in group II when compared to controls and other patient groups. This may be related to DFO induced reduction in iron overload or its direct effect on immunity [30].



When classifying the studied patients into splenectomized and non splenectomized groups; splenectomized patients had higher significant levels regarding serum ferritin,  $\mathrm{CD}3+,$ ,  $\mathrm{CD4+}$  and  $\mathrm{CD8+}$ compared to non splenectomized ones  $\mathrm{(p\leq~0.05)}$  which was in agreement with Gharagozloo et al. and Lee et al. [25,39]. Increased T lymphocyte counts in post-splenectomy might be associated with antigens that could not be filtered by the spleen. This suggests that the spleen could play a role in the regulation of lymphocytes [25]. Iron overload indicated by ferritin was higher on post-splenectomy group than non- splenectomy group. This condition was due to iron absorption in response to ineffective erythropoiesis [40]. Therefore, iron overload, both from increased iron absorption or chronic blood transfusions, may affect the regulation and redistribution of lymphocyte subsets from the spleen and lymph nodes to the circulating pool and vice versa [41].

T-cells are required for the normal function of B-cells. Accordingly, most T-cells deregulation affects B lymphocytes. Thus, we measured  $\mathrm{CD19+}$ B cells. As well, we measured  $\mathrm{CD56+}$ cell counts as an indicator of NK cells. We revealed that percentage of  $\mathrm{CD19+}$ cells was significantly higher in all patients groups compared with controls  $\mathrm{(p=0.01)}$ . This was in accord with AI- Consolini et al. and Pattanapanyassat et al. [40,42] but Ahmadiafshar et al. and Gharagozloo et al. found insignificant statistical differences regarding  $\mathrm{CD19+}$  cells [25,43]. Also, we found that the percentages of  $\mathrm{CD56+}$  cells was significantly lower in group III compared with group I, II and controls. This matched the findings in Hagag et al. and Ahmadiafshar et al. [43,44]. The depressed natural killer cell activity in  $\upbeta\mathrm{TM}$ patients may follow repeated blood transfusions and iron overload. Accordingly, this raises the concern of reduced resistance to viral infections and to the development of malignancy [45]. This makes thalassaemia patients susceptible to an increased risk of serious systemic infections [46]. Contrary, Gharagozloo et al. found no statistical difference in  $\mathrm{CD56+}$  cells between patient and controls [25].

These contradictory results between our study and the data of some previous studies could be referred to many factors. These factors are including: 1) Clinical heterogeneity, ethnicity and geographic regions variations because thalassaemia patients are different genetically and thus phenotypically due to human migrations. [47] 2) Frequency of blood transfusion [7,8,19,48,49]. Li et al. had reported that the number of NK cells is inversely proportional to transfusion numbers. 3) Body iron status and iron chelating therapy or 4) Duration since splenectomy [45].

The small size of the enrolled children is one of the limitations of this research which is related to the nature of this study as a region- specific one. This could be both limitation and advantage at the same time as this minimized the heterogeneity between individuals resulting in a more or less homogenous study groups. Thus, all patients had quite comparable predisposing risk factors for infections. More studies with greater multiregional populations seem to be required. Additionally, recognizing the genetic mutations of the enrolled patients plus correlating between these mutations and their phenotypic characters are fundamental. This will help in defining the role of location in  $\upbeta\mathrm{TM}$ -induced complications or related responses to different therapies. Furthermore, covering the remaining immunologic parameters which are lacking in our study because of financial issues should be performed. As a consequence of this study along with earlier ones, regular monitoring of the immune status of  $\upbeta\mathrm{TM}$ patients is recommended. This can increase our knowledge of their immune system behavior aiming to improve their quality of life and to prolong their survival rates.



## Conclusion

Our results support that thalassaemia patients are immunologically different from normal children and that the difference in treatment approaches are crucial players in this immune alteration. Also,  $\upbeta\mathrm{TM}.$ - related immune modulations have a range of regional heterogeneity. Thus, contrasting findings are expected and regular follow-up of βTM immune status is needed aiming to reduce their liability to catch infections.



## Ethical Considerations

The study was carried out in accordance with the World Medical Association’s Declaration of Helsinki and approved by the research ethics committee of Minia University. The rationale, nature and possible risks of the experiments were fully explained to the parents. All parents gave written, informed consents at the beginning of the study and all data were kept confidential and used for research purposes only.

## References

1.Cao A, Galanello R (2010) Beta-thalassemia. Genet Med 12: 61-76. 2.Alamiry A, Ali T, Majeed M (2011) Detection of Hemoglobinopathies in Hypochromic, Microcytic and Sickeled Cell Blood Films by Hemoglobin Electrophoresis. Thi-Qar Med J 5: 139-148.

3.Vento S, cainelli F, Cesario F (2006) Infections and thalassaemia. Lancet Infect Dis 6: 226-233.

4.Rahav G, Volach V, Shapiro M, Rund D, Rachmilewitz EA, et al. (2006) Severe infections in thalassemic patients: prevalence and predisposing factors. Br J Haematol 133: 667-674.

5.Tubman VN, Fung EB, Vogiatzi M, Thompson AA, Rogers ZR, et al. (2015) Guidelines for the Standard Monitoring of Patients with Thalassemia: Report of the Thalassemia Longitudinal Cohort. J Pediatr Hematol Oncol 37: 162-169.

6.Farmakis D, Giakoumis A, Polymeropoulos E, Aessopos A (2003) Pathogenetic aspects of immune deficiency associated with beta- thalassemia. Med Sci Monit 9: 19-22.

7.Capellini M, Cohen A, Eleftheriou A, Piga A, Porter J, et al. (2008) Guidelines for the clinical management of thalassemia 2008. (2nd revised edition), Nicosia.

8.Weatherall (2010) The inherited diseases of hemoglobin are an emerging global health burden. Blood 115: 4331-4336.

9.Morabiton N, Russogt GT, Gaudio A, Lasco A, Catalano A, et al. (2007) The “lively” cytokines network in beta-thalassemia major-related osteoporosis. Bone 40: 1588-1594.

10.Alidoost F, Gharagozloo M, Bagherpour B, Jafarian A, Saj-Jadi SE, et al. (2006) Effects of silymarin on the proliferation and glutathione levels of peripheral blood mononuclear cells from beta-thalassemia major patients. Int Immunopharmacol 6: 1305-1310.

11.Sari TT, Gatot D, Akib AA, Bardosono S, Hadinegoro SR, et al. (2014) Immune response of thalassemia major patients in Indonesia with and without splenectomy. Acta Med Indones 46: 217-225.

12.Higgs DR, Engel JD, Stamatoyannopoulos G (2012) Thalassaemia. Lancet 379: 373-383.

13.Gomes MP, Smedbol E, Chalifour A, Hénault-Ethier L, Labrecque M, et al. (2014) Alteration of plant physiology by glyphosate and its by-product aminomethylphosphonic acid: an overview. J Exp 65: 4691-4703.

14.Yu Q, Han H, Cawthray GR, Wang SF, Powles SB (2013) Enhanced rates of herbicide metabolism in low herbicide-dose selected resistant Lolium rigidum. Plant Cell Environ 36: 818-827.

15.Asadov ChD (2014) Immunologic Abnormalities in  $\upbeta$ -Thalassemia. J Blood Disorders Transf 5: 224.

16.Jeddoa, A, Saar JR, Adel AA (2011) Immunological Evaluation of Patients with Β-Thalassemia Major in Kerbala City Using Single Radial Immunodiffusion (SRID) Technique. Karbala J Med 4: 1-2.

17.Roos A, Bouwman LH, Van Gijlswijk-Janssen DJ, Faber-Krol MC, Stahl GL, et al. (2001) Human IgA activates the complement system via the mannan-binding lectin pathway. J Immunol 167: 2861-2868.

18.Salman AN, Al-Mousawi NH (2014) Evaluation of Some Humoral Immune System Parameters in Beta Thalassemia Major Patients at Wasit Province / Iraq. Journal of University of Thi-Qar 9: 88-94.

19.Ghaffari J, Vahidshahi K, Kosaryan M, Soltantooyeh Z, Mohamadi M (2011) Humeral immune system state in  $\upbeta$ thalassemia major. Med Glas Ljek Komore Zenicko-doboj. Thalassemia and immune system dysfunction-review article. Med Glas Ljek Komore Zenicko-doboj. Kananota 8: 192-196.

20.Ezer U, Gülderen F, Culha VK, Akgül N, Gürbüz O (2002) Immunological status of thalassemia syndrome. Pediatr Hematol Oncol 19: 51-58.

21.Darzi AA, Kamali S, Khakzad M (2015) Influence of splenectomy on immunoglobulins and complement components in major thalassemia. Caspian J Intern Med 6: 30-33.

22.Mahmood NS, Abbas AM, Latif II, Mohammed ZJ (2015) Qualitative evaluation of humoral immunity of  $\upbeta$ -thalassaemia patients by immunofixation test. International Journal Of Current Medical And Pharmaceutical Research 1: 50-53.

23.Amin AJ, Susan J, Reza A, Aale YS, Jamalian N, et al. (2005) Evaluation of serum levels of immunoglobulin and complement factors in  $\upbeta\cdot$ - thalassemia major patients in southern Iran. I journal immunology 4: 220-225.

24.Ahluwalia J, Datta U, Marwaha RK, Sehgal S (2000) Immune functions in splenectomised thalassemic children. Indian J Pediatr 67: 871-876.

25.Gharagozloo M, Karimi M, Amirghofran Z (2009) Double-faced cell- mediated immunity in beta-thalassemia major: stimulated phenotype versus suppressed activity. Ann Hematol 88: 21-27.

26.Javad G, Saeid A, Mohammadmehdi N (2011) Thalassemia and immune system dysfunction-review article. Int J Curr Res 3: 105-108.

27.Brand A (2002) Immunological aspects of blood transfusions. Transplant Immunology 10: 183-190.

28.Abbas A, Andrew H, Pilai S (2007) Cellular and molecular immunology. Philadelphia 2007 PA 19103-2899 SECTION II Recognion of An gens 77-92.

29.El Yazji MS (2011) Immunological assessment of beta-thalassemia major children aged 5-12 years old attending Abd El-Aziz El-Rantisy Hospital in Gaza strip. TURK J IMMUNOL 16: 1.

30.Aleem A, Shakoor Z, Alsaleh K, Algahtani F, Iqbal Z, et al. (2014) Immunological evaluation of  $\upbeta$ -thalassemia major patients receiving deferasirox. Journal of the College of Physicians and Surgeons Pakistan 24: 467-471.

31.Kiani-Amin M, Daneshi M, Ayazi P, Mohammadian S, Rezaei N (2011) Serum immunoglobulin levels in splenectomized and non- splenectomized patients with major Beta-thalassemia. Iran J Pediatr 21: 95-98.

32.Waghorn DJ (2001) Overwhelming infection in asplenic patients: current best practice preventive measures are not being followed. J Clin Pathol 54: 214-218.

33.Moshtaghi-kashanian GR, Gholamhoseinian A, Hoseinimoghadam A, Rajabalian S (2006) Splenectomy changes the pattern of cytokine production in beta-thalassemic patients. Cytokine 35: 253-257.

34.Shete A, Thakar M, Abraham PR, Paranjape R (2010) A review on peripheral blood  $\mathrm{CD4+\ensuremath{\,\mathrm{~T~}\,}~}$  lymphocyte counts in healthy adult Indians. Indian J Med Res 132: 667-675.

35.Bains I, Antia R, Callard R,Yates AJ (2009) Quantifying the development of the peripheral naive  $\mathrm{CD4+}$ T-cell pool in humans. Blood 113: 5480-5487.

36.Kadam PP, Manglani MV, Sharma SM, Sharma RA, Setia MS (2014) Immunoglobulin Levels and CD4 / CD8 Counts in Thalassemia Major. Indian pediatric 8: 1000-1002.

37.Noulsri E, Lerdwana S, Fucharden S, Pattanapanyasat K (2014) Phenotypic characterization of circulating CD4/CD8 T-lymphocytes in beta thalassemia patients. Asian Pac J Allergy Immunol 32: 261-269.

38.Del Vecchio GC, Schettini F, Piacente L, De Santis A, Giordano P, (2002) Effects of deferiprone on immune status and cytokine pattern in thalassaemia major. Acta Haematologica 108: 144-149.

39.Lee AC, Li CH (2004) Age as a factor in severe bacterial infection in transfusion-dependent patients with thalassemia major. Clin InfectDis 38: 1194-1195.

40.Pattanapanyasat K, Thepthai C, Lam-chiagdhase P, Lerdwana S, Tachava- nich K, et al. (2000) Lymphocyte subsets and specific T-cell immune response in thalassemia. Cytometry 42: 11-17.

41.Cunningham-Rundles S, Giardina PJ, Grady RW, Califano C, McKenzie P, et al. (2000) Effects of transfusional iron overload on immuneresponse. The Journal of Infectious Diseases 182: S115-S121.

42.Consolini R, Calleri A, Legitimo A, Massei F (2001) Immunological evaluation of patients with beta-thalassemia major. Acta Haematol 105: 7-12.

43.Ahmadiafshar A, Ghadipasha A, Mousavinasab N (2012) Immunological evaluation in patients with beta thalassemia major. Arch Dis Child 97: A217.

44.AA Hagag, A Mohamed, ES Elgamasy, A Elbar (2015) Assessment of T lymphocyte subsets in children with beta thalassemia major with iron overload. Egypt J Pediatr Allergy Immunol 13: 57-63.

45.Li  $\mathrm{K},$ Li CK, Wong RP, Wong A, Shing MM, et al. (1997) Transfusion related immunomodulation in Chinese children with thalassemia. Vox Sang 73: 167-173.

46.Speer CP, Gahr M, Schuff-Werner P, Schroter W (1990) Immunologic evaluation of children with homozygous beta-thalassaemia treated with desferrioxamine. ActaHaematol 83: 76-81.

47.Guglielmo P, Cunsolo F, Lombardo T, Sortin G, Giustolisi R, et al. (1984) T-subset abnormalities in thalassaemia intermedia: Possible evidence for a thymus functional deficiency. Acta Haematol 72: 361-367.

48.Singer S, Wu V, Mignacca R (2000) Alloimmunization and erythrocyte autoimmunization in transfusion-dependent thalassemia patients of predominantly. Asian Descent. Blood 96: 3369-3373.

49.Rachmilewitz EA, Giardina PJ (2011) How I treat thalassemia. Blood 118: 3479-3488.
# Comparison of the Immunity Status in-Between Children with  $\upbeta\cdot$ -Thalassaemia Major Receiving Different Treatment Modalities: A Single Egyptian District Study

Mahmoud SS, Mohamed GB, Hakeem GLA, Higazi AM2, Nafady AAH, Farag NM2, Mahrous DM1, Kamal NN4, Hassan Z1, Aziz EA1 and Shaban A1 Department of Pediatrics, Faculty of Medicine, Minia University, Minia, Egypt Department of Clinical Pathology, Faculty of Medicine, Minia University, Minia, Egypt Department of Clinical Pathology, Faculty of Medicine, Assuit University, Assuit, Egypt Department of Public Health, Faculty of Medicine, Minia University, Minia, Egypt

   

  

 



\*Corresponding author: Gehan Lotfy Abdel Hakeem, Department of Pediatrics, Faculty of Medicine, Minia University, Minia, Egypt, Tel: +201001818254; E-mail: gehanlotfy72@yahoo.com Received date: November 29, 2016; Accepted date: January 08, 2017; Published date: January 16, 2017 Copyright:  $\copyright2017$ Mahmoud SS, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

 



### Abstract

Background:  $\upbeta$ -thalassaemia major is one of the chronic hemolytic anemias resulting from defect in  $\upbeta\cdot$ -globin chain. It requires frequent blood transfusion plus other treatment modalities. These treatment modalities may be associated with certain immunologic modulations.

Objective: To assess the immunity status in children with  $\upbeta$ -thalassaemia major under different treatment regimens within El Minia, Egypt.

Subjects and Methods: One hundred forty-four children were enrolled and classified into four groups. Thirty-six  $\upbeta$ -thalassaemia patients treated only with blood transfusion (group I). Thirty-six patients treated with transfusion and iron chelation (group II). Thirty-six patients treated with transfusion, iron chelation and subjected to splenectomy (group III). Group IV involved thirty-six apparently healthy age and sex matched children. CBC plus serum levels of ferritin, IgA, complement C3 and C4 were measured along with detection of  $^{\mathrm{CD3+}}$ ,  $C D4+$ ,  $\mathsf{C D8+}$ ,  $\mathsf{C D}\,19+$ and  $\mathsf{C D56+}$  lymphocyte percentages and absolute counts.

Results: IgA levels were significantly higher in thalassaemia patients compared to controls  $(\mathsf{p}\!\!<\!0.001)$  plus highly significant increase in IgA levels in splenectomized patients than non-splenectomized  $(\mathsf{p}\!\!<\!0.001)$ . Levels of C3 were significantly decreased in all patients compared with controls  $(\mathsf{p}\mathrm{=}0.001)$  with a highly significant decrease in C3 levels in splenectomized patients than non splenectomized ones  $(\mathsf{p}\!\!<\!0.001)$  but no statistical difference between their C4 levels. Significant statistical differences were revealed regarding  ${\mathsf{C D3+}}$ ,  $C D4+$ and  $^{\mathrm{CD8+}}$ T lymphocyte percentages within thalassaemia groups when compared to each other’s and to controls. Splenectomized patients had higher significant levels regarding serum ferritin  $(\mathsf{p}\mathsf{=}0.02)$  along with  $^{\mathrm{CD3+}}$ ( $\mathsf{p}\mathrm{=}0.05)$ ,  $\mathsf{C D4}+(\mathsf{p}\mathsf{=}0.05)$ and  $\mathsf{C D8+}$   $(\mathsf{p}\mathrm{=}0.037)$  lymphocyte percentages compared to non-splenectomized.  $\mathsf{C D}\,19+$ lymphocyte percentages were significantly higher while  $\mathsf{C D56+}$ lymphocyte percentages were significantly lower in all patients compared with controls  $(\mathsf{p}\!\!=\!\!0.02$  and 0.05).

Keywords: Thalassaemia; Immunity; Treatment modalities; Egyptian children; El Minia region Abbreviations: CBC-Complete Blood Count; IgA-Immunoglobulin A



## Introduction

Thalassaemia is defined as a group of inherited disorders that arise as a result of certain mutations in hemoglobin (Hb) genes.  $\upbeta\cdot$ - thalassaemia major (βTM) has a high prevalence in the Mediterranean region, including Egypt, Middle East, Indian subcontinent, and South East Asia [1]. Though, it is a growing global health problem due to extensive population migrations. About  $1.5\%$  of the world populations are carriers of  $\upbeta$ -thalassaemia gene [2]. βTM patients present with many problems rather than severe anemia, including increased susceptibility to infections which constitutes the second most common cause of mortality and a major cause of morbidity in  $\upbeta$ -thalassaemia after heart failure [3]. The underlying causes of their increased liability to infections may be attributed to anemia per-se, reticuloendothelial system dysfunction as well as therapeutic regimens-related infections [4].



These therapeutic approaches incorporate repeated blood transfusion alone or in combination with iron chelators. Splenectomy is added when indicated. Recently, gene therapy and bone marrow transplant or stem cell therapy are promising hopes [5]. Volumetric and multiple transfusions lead to alloimmunization, accumulation of iron in different tissues and are associated with increased risk of infections. These infections are caused by lots of bacterial and viral infections such as cytomegalo virus, Ebstein-Barr virus and hepatitis C virus [6]. Likewise, Yersinia enterocolitica is usually associated with the use of deferoxamine (DFO) as an iron chelator [7]. As well, the risk of sepsis in splenectomized patients is as high as  $7\%$ over a 10-years period and almost  $25\%$  of splenectomized patients are at the risk of severe infections [8]. Hence, it is crucial to identify the impact of these treatment modalities on the immune status of these patients.



Several studies about the immune competence in  $\upbeta$ -thalassaemia had revealed numerous quantitative and functional defects, involving T and B lymphocytes, immunoglobulins in addition to the impairment in components of the complement system [9,10]. Additionally, immunologic disorders in patients with  $\upbeta\mathrm{TM}$ comprise decreased absorption and phagocytic ability of segmented neutrophils, changes in cytokines production as well as dysfunction of macrophages, properdin and lysozymes [11]. As well, it has been reported that the distribution of  $\upbeta$ -globin gene mutations differs among ethnic groups plus its regional and individual variation [12]. Furthermore, there is increasing evidence suggesting that clinical features or complications of  $\upbeta\mathrm{TM}$  plus their immune characters and even their response to various therapies are population dependence. Also, the immunity status of  $\upbeta\mathrm{TM}$  patients varies in relation to the type and extent of treatment approaches [8].

The overall objective of this study was to identify the immunological alterations of  $\upbeta\mathrm{TM}$  patients within Egyptian children in El Minia district. Our aim was to compare the immune competence between those  $\upbeta\mathrm{TM}$  children who receiving multiple blood transfusions alone or with iron chelating agents plus or minus splenectomy. We studied serum immunoglobulin A (IgA) levels beside serum C3 and C4 complements in addition to blood counts of T, B and natural killer (NK) lymphocytes.

## Subjects and Methods

### Subjects

This multicenter, case-control study was conducted in El Minia city in Egypt. Patients were selected from Pediatric Hematology Outpatient Clinic at Minia University Hospital and from Pediatric Hematology Outpatient Clinic, National Blood Bank, Minia Ministry of Health while controls were selected from apparently healthy children from Pediatric Growth Clinic, Minia University Hospital. This study was carried out between May and December, 2015. One hundred and forty-four children were included in this study, which involved a group of 108 thalassaemia patients and 36 apparently healthy children with matching age and sex as a control group IV. Control children were 18 males  $(50\%)$  and 18 females  $(50\%)$ . There ages were ranged from 2-16 years old. The thalassaemia patients group was further subdivided into three subgroups according to the difference in their treatment regimens. These subgroups comprised patient groups from I-III. Group I patients (No. 36) were receiving blood transfusion only. These patients were 18 males  $(50\%)$  and 18 females  $(50\%)$ . There ages were ranged from 2-8 years old. Patients in group II (No. 36) were receiving blood transfusion and iron chelation while patients in group III (No. 36) were receiving blood transfusion plus iron chelation and also subjected to splenectomy. Group II patients were 12 males  $(33.3\%)$  and 24 females  $(66.7\%)$ . There ages were ranged from 5-12 years old. Group III patients were 21 males  $(58.3\%)$  and 15 females  $(41.7\%)$ . There ages were ranged from 10-16 years old.



All included patients and controls were subjected to complete history taking (name, age, sex, residence, family history, consanguinity, age of 1st blood transfusion, amount of transfusion per year, duration of splenectomy in splenectomized patients in addition to history of exposure to infections). Moreover, the involved children were examined carefully (general examination, anthropometric measures which were plotted on percentile growth charts, vital data as well as examination of chest, heart and abdomen). The exclusion criteria were patients who had a bone marrow transplant. Finally, the laboratory investigations were performed to all subjects. These investigations were comprised Complete Blood Count (CBC), blood smears, Hemoglobin  $\mathrm{(Hb)}$  electrophoresis for controls, serum ferritin, serum level of immunoglobulin A (IgA), complements C3 and C4 plus percentages and absolute counts of CD3, 4, 8, 19 and 56 T lymphocytes.

### Blood Sampling

Venous blood samples were collected from both patients and controls under complete aseptic conditions. About  $2\;\mathrm{ml}$ of blood were withdrawn in K3-EDTA anticoagulant tubes for complete blood counts, peripheral blood smears as well as flow cytometric analysis (immunophenotyping). About 3ml of blood were withdrawn in a plain tube without any anticoagulant. Their serum was then separated and stored at -  $70^{\circ}\mathrm{C}$ till used for serum ferritin, IgA and complements assays.

Blood samples were withdrawn from thalassaemia patients just prior to a scheduled transfusion. The blood samples from the control group were taken while coming for follow-up of their growth. All controls had normal Hb levels for their age and sex with normal red blood cell indices. At the time of sampling all patients and controls were apparently free from infections (normal C-reactive protein).

### Laboratory methodology

Complete blood count was performed using an automated blood counter (Sysmex KX-21N). Additionally, peripheral blood smears were stained by Leishman stain. Reticulocyte counts were detected as well. The absolute lymphocytes and neutrophils counts were calculated. Serum ferritin levels were measured using human in vitro Enzyme- Linked Immunosorbent Assay (EIA) kit from Abcam Inc, Cambridge, USA. The assay was performed according to the manufacturer’s instructions. Ferritin values were expressed as nanogram per milliliter  $\left(\mathrm{ng/ml}\right)$ . [13,14] Levels of IgA were determined using human in vitro EIA kit from Abcam Inc, Cambridge, USA according to the manufacturer’s instructions. Final IgA values were expressed in milligram per deciliter  $(\mathrm{mg/dl})$ . As well, Complements C3 and C4 serum levels were detected with human in vitro EIA kit from Abcam Inc, Cambridge, USA. Complements C3 and C4 values were expressed in milligram per deciliter  $(\mathrm{mg/dl})$ .



### Flow cytometric immunophenotypic analysis

T- Lymphocyte subsets in whole blood samples were enumerated using fluoroisothiocyanate (FITC) conjugated CD4 (Becton Dickinson, Bioscience, USA), phycoerythrin (PE) conjugated CD8 (Becton Dickinson, Bioscience, USA) and peridinium-chlorophyll- protein (Per-CP) conjugated CD3 (Becton Dickinson, Bioscience, USA). B-Lymphocyte subsets in whole blood samples were enumerated using phycoerythrin (PE) conjugated CD19 (Becton Dickinson, Bioscience, USA). Natural killer Lymphocyte subsets in whole blood samples were enumerated using phycoerythrin (PE) conjugated CD56 (Becton Dickinson, Bioscience, USA). Flow cytometric analysis was done by FACS Calibur flow cytometry with Cell Quest software (Becton Dickinson Biosciences, USA). An isotype matched negative control was used with each sample. Forward and side scatter histogram was used to define the lymphocyte population (R1). The absolute counts and percentages of CD3, 4, 8, 19 and 56 subsets of lymphocytes were calculated.

 



### Statistical analysis

The clinical and laboratory data were recorded. These data were analyzed using statistical program for social science SPSS software version 17 (SPSS Inc., Chicago, IL, USA). Quantitative variables were presented as mean  $\pm$ standard deviation (SD). On the other hand, description of qualitative variables was presented as number (No.) and percentage  $(\%)$ . Results were expressed as tables and figures. Graphics were done by Excel, Microsoft Office 2010. Student t-test was used to compare results between groups as regards quantitative variables. The correlation was performed by using Pearson correlation coefficient (r). For all tests, a probability (p) was considered non-significant if  ${>}0.05.$ , significant if  $\leq0.05$  and highly significant if  $\leq0.001$ . One Way ANOVA test was used for comparison of quantitative data between more than two groups. The Chi-square test was used to compare qualitative variables between groups.



## Results

### Comparison of laboratory results and immunological parameters between  $\upbeta\mathrm{.}$ -thalassaemia major children under different treatment modalities and controls

A significant difference was found on comparison between all thalassaemia patients (group I, II, III) and control group regarding Hb levels  $\scriptstyle(\mathrm{p}=0.015)$ , ANC  $\mathrm{(p=0.05)}$ , serum IgA  $(\mathrm{p}{<}0.001)$ , serum C3  $\scriptstyle(\mathrm{p}=0.001)$ , serum C4 ( $\mathrm{\Deltap=}0.002\mathrm{\rangle}$ ,  $\mathrm{CD}3+$  absolute count  $\mathrm{(p=0.05)}$ ,  $\mathrm{CD3+/}$   $\mathrm{CD4+}$  absolute count  $\mathrm{(p=0.04)}$ ,  $\mathrm{CD3+/CD8+}$  absolute count  $\mathrm{(p=0.06)}$ ,  $\mathrm{CD19+}$  absolute count  $\mathrm{(p=0.02)}$  and  $\mathrm{CD56+}$ absolute count  $\mathrm{(p=0.05)}$  (Table 1).

Also, differences within the 3 thalassaemia groups regarding IgA, C3, C4 (Table 2) along with percents of  $\mathrm{CD}3+$ ,  $\mathrm{CD3+/CD4+},$ ,  $\mathrm{CD}3+/$   $\mathrm{CD8+},$ ,  $\mathrm{CD19+}$  and  $\mathrm{CD56+}$  as well as their absolute counts were summarized (Figure 1 and Table 2).

Table 2: Comparison between studied patients regarding immunological markers; Group I: thalassaemia children receiving blood transfusion only; Group II: thalassaemia children receiving blood transfusion  $^+$ iron chelation; Group III: thalassaemia children receiving blood transfusion  $^+$ iron chelation  $^+$ splenectomy; vs: versus; Ig A: Immunoglobulin A; C3: Complement C3; C4: Complement C4; Non sig.  ${>}0.05.$ , Significant $\leq0.05^{\star};$ , High Significant  $\leq0.001^{\ast\ast}$ .

non splenectomized groups  $(\mathrm{p}{<}0.001)$  (Figure 2A). Moreover, splenectomized patients showed significant decrease in serum C3 levels  $_{(\mathrm{p<0.001})}$  and non-significant difference in serum C4 levels in comparison to non splenectomized groups  $\scriptstyle(\mathrm{p}=0.282)$  (Figure 2B).

 $\mathrm{CD}3+,$   $^{4+}$  and  $^{8+}$ percentages were statistically significant higher in the splenectomized group in comparison to non splenectomized patients  $\scriptstyle(\mathrm{p}=0.05,\ 0.05$ and 0.037 respectively). On the other hand, splenectomized children showed non-significant difference from non splenectomized ones concerning  $\mathrm{CD19+}$ percentage, but highly statistically significant lower levels regarding  $\mathrm{CD56+}$ ( $\mathrm{{p=}}0.235$ , <0.001 respectively) (Figure 2C).

### Serum ferritin levels within  $\upbeta.$ -thalassaemia subgroups in correlation to immunological characteristics

The mean  $\pm\ \mathrm{SD}$ of serum ferritin levels within thalassaemia groups as well as controls were shown in table 1. There was a significant increase in serum ferritin levels when patients were compared with controls  $\scriptstyle(\mathrm{p}=0.013)$ . Additionally, serum ferritin levels were elevated in splenectomized patients (group III) compared with non splenectomized ones  $\mathrm{(p=0.02)}$  (Figure 3). Significant moderate positive correlation was found between  $\mathrm{CD}3+$ cells and ferritin  $\mathrm{(p=0.04,r=0.6)}$ . As well, significant fair positive correlation was found between  $\mathrm{CD3+/}$   $\mathrm{CD8+}$  cells and ferritin  $\mathrm{(p=0.04,r=}0.3\mathrm{)}$  in group I. There were no other significant correlations between ferritin and the remaining immune parameters (Table 3).

The comparison between splenectomized patients (group III) and non splenectomized ones (group I plus II) concerning the studied immune parameters was illustrated in Figure 1. Serum IgA levels were statistically significant high in splenectomized patients compared with

## Discussion

It had been shown that the immune-capability of  $\upbeta\mathrm{TM}$ patients has been altered [9, 10]. The presence of abnormal erythrocytes leads to continuous activation of monocytes and immune clearance [15]. Furthermore, it has been shown that  $\upbeta\mathrm{TM}$ -related immunologic changes differ according to the variety and duration of therapeutic procedures and in a population dependence manner [8]. Data relevant to the impact of different treatment modalities on the immune system in thalassaemia patients are yet limited as these data should be adjusted for geographical differences. The aim of this work was to evaluate the immune status of Egyptian children within El Minia region who suffering from  $\upbeta\mathrm{TM}$ and receiving various treatment modalities.

Immunoglobulins and complement proteins play a pivotal role in humoral immunity. Therefore, we assessed the deviations of these parameters among thalassaemia patients according to their treatment regimens [16]. This study has been concerned with assessment of IgA, serum complement C3 and C4 levels. IgA is an important mediator of mucosal immunity along with its involvement in complement activation via alternative and lectin pathways. Additionally, complement C3 and C4 are among the most commonly measured complement components [17].

The present study demonstrated that serum IgA levels were significantly higher in all groups of  $\upbeta\mathrm{TM}$  compared to healthy controls  $(\mathrm{p}{<}0.001)$ . This was similar to what was detected by Ali N. Salman, et al., Jeddoa et al. and Ghaffari et al. [16,18,19]. On the other hand, Ezer et al. showed that serum levels of IgA were normal within all subgroups of βTM. In Ezer study, all patients with βTM appeared to have an intact humoral immune system whatever their criteria. In addition, our data showed that splenectomized patients had higher significant increase in serum IgA levels than non splenectomized ones  $(\mathrm{p}{<}0.001)$  [20]. Many previous studies had supported our data, but Ahluwalia et al. didn’t detect any impacts of splenectomy on immunoglobulins [21-24].



The mechanisms of such increase in serum IgA levels can be attributed to iron overload in the skin of thalassaemia patients which leads to stimulation of the mucocutaneous IgA production [25]. As well, it has been suggested that iron overload causes an increased migration of T helper cells to the gut and lymph nodes which increases serum immunoglobulins levels especially IgA as the main immunoglobulin isotype in most mucosal surfaces [18]. Also, iron and its binding proteins have immune regulatory properties. Therefore, iron excess may tip the immune balance unfavorably to allow increased growth rates of infectious organisms followed by increased immunoglobulin levels [8]. Moreover, repetitive transfusions lead to continuous alloantigenic stimulation [26,27]. Additionally, thalassaemia patients are prone to many bacterial and viral infections [3]. Therefore, the immune system is activated and immunoglobulin levels are elevated [8]. Finally, increased serum levels of IgA mostly after splenectomy could be due to the pressure that occurs on other secondary lymphoid organs to synthesis major immunoglobulin classes for compensation of spleen loss [28].



In the current study, the mean levels of complements C3 and C4 were significantly decreased in all patient groups compared with controls  $\scriptstyle(\mathrm{p}=0.001\$ , 0.002 respectively). This is in accordance with Jeddoa et al., El Yazji and Amin et al. studies. [16,23,29] Ezer et al. results are unlike ours as their study showed no significant difference in C3 levels between patients and controls [20]. This may be explained on the basis that repeated blood transfusion could result in continuous exposure to various antigens which lead to continuous immune system stimulation and hence complement consumption [8]. The addition of iron chelators in our study led to significant elevation of complement C3 and C4 along with increased serum IgA levels in Group II than Group I ( $\mathrm{\Deltap=}0.01$ , 0.03 and 0.04 respectively). Aleem A et al. showed significant immune abnormalities in  $\upbeta\mathrm{TM}$ patients with oral iron chelators including low C4 level [30]. As well, we found a highly significant decrease in serum C3 levels in splenectomized patients than non splenectomized ones  $(\mathrm{p}{<}0.001)$ . This was the same as Darzi AA et al. results [21] and unlike Kiani-Amin et al. outcomes [31]. The spleen plays a role in releasing starter proteins of complement activation pathways. So, splenectomy causes reduction in complement components [21]. Also, splenectomized patients have the most increased risk of overwhelming infections which stimulate the immune system, leading to complement consumption [23,32].

Lymphocytes are crucial in cell mediated immunity. Lymphocyte subpopulations include T cells, B cells and NK cells. T lymphocytes contain  $\mathrm{CD4+}$ T cells, which become activated upon foreign antigen exposure, such as intracellular pathogens, fungi and protozoa in addition to  $\mathrm{CD8+}$ T-cells which destroy virally infected cells.  $\mathrm{CD4+}$  cells along with  $\mathrm{CD8+}$ cells represent the majority of T lymphocytes [33]. We estimated the absolute counts along with percentages of  $\mathrm{CD}3+$ lymphocytes as representative of total T Lymphocytes (pan-T cell marker) besides  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ T subsets. Since absolute counts can vary from day to day and according to child age, it is more useful to look at the percentages of these lymphocyte subpopulations [34,35]. We found significant statistical differences regarding percentages of  $\mathrm{CD}3+,$   $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ cells within thalassaemia groups when compared to each other’s and to healthy controls. Patients of group I had significantly elevated  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages, when compared to controls and to Group II patients. Group II patients had significantly lower  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages when compared to other thalassaemia groups but group III had significantly elevated  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ percentages when compared to controls and other thalassaemia groups. These findings were in agreement with Kadam et al., Gharagozloo et al. and Vento et al. [3,25,36]. In contrast, Noulsri, et al. and Del Vecchio et al. had reported an insignificant difference in  $\mathrm{CD}3+$ cells and in its  $\mathrm{CD4+}$ and  $\mathrm{CD8+}$ subsets between patients and controls [37,38]. Blood transfusion causes chronic infections and thus constant stimulation of the immune system which might induce T cells. Besides, the majority of patients in this study were found to have increased  $\mathrm{CD4+}$  and  $\mathrm{CD8+}$ cell percentages except those in group II when compared to controls and other patient groups. This may be related to DFO induced reduction in iron overload or its direct effect on immunity [30].



When classifying the studied patients into splenectomized and non splenectomized groups; splenectomized patients had higher significant levels regarding serum ferritin,  $\mathrm{CD}3+,$ ,  $\mathrm{CD4+}$  and  $\mathrm{CD8+}$ compared to non splenectomized ones  $\mathrm{(p\leq~0.05)}$  which was in agreement with Gharagozloo et al. and Lee et al. [25,39]. Increased T lymphocyte counts in post-splenectomy might be associated with antigens that could not be filtered by the spleen. This suggests that the spleen could play a role in the regulation of lymphocytes [25]. Iron overload indicated by ferritin was higher on post-splenectomy group than non- splenectomy group. This condition was due to iron absorption in response to ineffective erythropoiesis [40]. Therefore, iron overload, both from increased iron absorption or chronic blood transfusions, may affect the regulation and redistribution of lymphocyte subsets from the spleen and lymph nodes to the circulating pool and vice versa [41].

T-cells are required for the normal function of B-cells. Accordingly, most T-cells deregulation affects B lymphocytes. Thus, we measured  $\mathrm{CD19+}$ B cells. As well, we measured  $\mathrm{CD56+}$ cell counts as an indicator of NK cells. We revealed that percentage of  $\mathrm{CD19+}$ cells was significantly higher in all patients groups compared with controls  $\mathrm{(p=0.01)}$ . This was in accord with AI- Consolini et al. and Pattanapanyassat et al. [40,42] but Ahmadiafshar et al. and Gharagozloo et al. found insignificant statistical differences regarding  $\mathrm{CD19+}$  cells [25,43]. Also, we found that the percentages of  $\mathrm{CD56+}$  cells was significantly lower in group III compared with group I, II and controls. This matched the findings in Hagag et al. and Ahmadiafshar et al. [43,44]. The depressed natural killer cell activity in  $\upbeta\mathrm{TM}$ patients may follow repeated blood transfusions and iron overload. Accordingly, this raises the concern of reduced resistance to viral infections and to the development of malignancy [45]. This makes thalassaemia patients susceptible to an increased risk of serious systemic infections [46]. Contrary, Gharagozloo et al. found no statistical difference in  $\mathrm{CD56+}$  cells between patient and controls [25].

These contradictory results between our study and the data of some previous studies could be referred to many factors. These factors are including: 1) Clinical heterogeneity, ethnicity and geographic regions variations because thalassaemia patients are different genetically and thus phenotypically due to human migrations. [47] 2) Frequency of blood transfusion [7,8,19,48,49]. Li et al. had reported that the number of NK cells is inversely proportional to transfusion numbers. 3) Body iron status and iron chelating therapy or 4) Duration since splenectomy [45].

The small size of the enrolled children is one of the limitations of this research which is related to the nature of this study as a region- specific one. This could be both limitation and advantage at the same time as this minimized the heterogeneity between individuals resulting in a more or less homogenous study groups. Thus, all patients had quite comparable predisposing risk factors for infections. More studies with greater multiregional populations seem to be required. Additionally, recognizing the genetic mutations of the enrolled patients plus correlating between these mutations and their phenotypic characters are fundamental. This will help in defining the role of location in  $\upbeta\mathrm{TM}$ -induced complications or related responses to different therapies. Furthermore, covering the remaining immunologic parameters which are lacking in our study because of financial issues should be performed. As a consequence of this study along with earlier ones, regular monitoring of the immune status of  $\upbeta\mathrm{TM}$ patients is recommended. This can increase our knowledge of their immune system behavior aiming to improve their quality of life and to prolong their survival rates.



## Conclusion

Our results support that thalassaemia patients are immunologically different from normal children and that the difference in treatment approaches are crucial players in this immune alteration. Also,  $\upbeta\mathrm{TM}.$ - related immune modulations have a range of regional heterogeneity. Thus, contrasting findings are expected and regular follow-up of βTM immune status is needed aiming to reduce their liability to catch infections.



## Ethical Considerations

The study was carried out in accordance with the World Medical Association’s Declaration of Helsinki and approved by the research ethics committee of Minia University. The rationale, nature and possible risks of the experiments were fully explained to the parents. All parents gave written, informed consents at the beginning of the study and all data were kept confidential and used for research purposes only.

## References

1.Cao A, Galanello R (2010) Beta-thalassemia. Genet Med 12: 61-76. 2.Alamiry A, Ali T, Majeed M (2011) Detection of Hemoglobinopathies in Hypochromic, Microcytic and Sickeled Cell Blood Films by Hemoglobin Electrophoresis. Thi-Qar Med J 5: 139-148.

3.Vento S, cainelli F, Cesario F (2006) Infections and thalassaemia. Lancet Infect Dis 6: 226-233.

4.Rahav G, Volach V, Shapiro M, Rund D, Rachmilewitz EA, et al. (2006) Severe infections in thalassemic patients: prevalence and predisposing factors. Br J Haematol 133: 667-674.

5.Tubman VN, Fung EB, Vogiatzi M, Thompson AA, Rogers ZR, et al. (2015) Guidelines for the Standard Monitoring of Patients with Thalassemia: Report of the Thalassemia Longitudinal Cohort. J Pediatr Hematol Oncol 37: 162-169.

6.Farmakis D, Giakoumis A, Polymeropoulos E, Aessopos A (2003) Pathogenetic aspects of immune deficiency associated with beta- thalassemia. Med Sci Monit 9: 19-22.

7.Capellini M, Cohen A, Eleftheriou A, Piga A, Porter J, et al. (2008) Guidelines for the clinical management of thalassemia 2008. (2nd revised edition), Nicosia.

8.Weatherall (2010) The inherited diseases of hemoglobin are an emerging global health burden. Blood 115: 4331-4336.

9.Morabiton N, Russogt GT, Gaudio A, Lasco A, Catalano A, et al. (2007) The “lively” cytokines network in beta-thalassemia major-related osteoporosis. Bone 40: 1588-1594.

10.Alidoost F, Gharagozloo M, Bagherpour B, Jafarian A, Saj-Jadi SE, et al. (2006) Effects of silymarin on the proliferation and glutathione levels of peripheral blood mononuclear cells from beta-thalassemia major patients. Int Immunopharmacol 6: 1305-1310.

11.Sari TT, Gatot D, Akib AA, Bardosono S, Hadinegoro SR, et al. (2014) Immune response of thalassemia major patients in Indonesia with and without splenectomy. Acta Med Indones 46: 217-225.

12.Higgs DR, Engel JD, Stamatoyannopoulos G (2012) Thalassaemia. Lancet 379: 373-383.

13.Gomes MP, Smedbol E, Chalifour A, Hénault-Ethier L, Labrecque M, et al. (2014) Alteration of plant physiology by glyphosate and its by-product aminomethylphosphonic acid: an overview. J Exp 65: 4691-4703.

14.Yu Q, Han H, Cawthray GR, Wang SF, Powles SB (2013) Enhanced rates of herbicide metabolism in low herbicide-dose selected resistant Lolium rigidum. Plant Cell Environ 36: 818-827.

15.Asadov ChD (2014) Immunologic Abnormalities in  $\upbeta$ -Thalassemia. J Blood Disorders Transf 5: 224.

16.Jeddoa, A, Saar JR, Adel AA (2011) Immunological Evaluation of Patients with Β-Thalassemia Major in Kerbala City Using Single Radial Immunodiffusion (SRID) Technique. Karbala J Med 4: 1-2.

17.Roos A, Bouwman LH, Van Gijlswijk-Janssen DJ, Faber-Krol MC, Stahl GL, et al. (2001) Human IgA activates the complement system via the mannan-binding lectin pathway. J Immunol 167: 2861-2868.

18.Salman AN, Al-Mousawi NH (2014) Evaluation of Some Humoral Immune System Parameters in Beta Thalassemia Major Patients at Wasit Province / Iraq. Journal of University of Thi-Qar 9: 88-94.

19.Ghaffari J, Vahidshahi K, Kosaryan M, Soltantooyeh Z, Mohamadi M (2011) Humeral immune system state in  $\upbeta$ thalassemia major. Med Glas Ljek Komore Zenicko-doboj. Thalassemia and immune system dysfunction-review article. Med Glas Ljek Komore Zenicko-doboj. Kananota 8: 192-196.

20.Ezer U, Gülderen F, Culha VK, Akgül N, Gürbüz O (2002) Immunological status of thalassemia syndrome. Pediatr Hematol Oncol 19: 51-58.

21.Darzi AA, Kamali S, Khakzad M (2015) Influence of splenectomy on immunoglobulins and complement components in major thalassemia. Caspian J Intern Med 6: 30-33.

22.Mahmood NS, Abbas AM, Latif II, Mohammed ZJ (2015) Qualitative evaluation of humoral immunity of  $\upbeta$ -thalassaemia patients by immunofixation test. International Journal Of Current Medical And Pharmaceutical Research 1: 50-53.

23.Amin AJ, Susan J, Reza A, Aale YS, Jamalian N, et al. (2005) Evaluation of serum levels of immunoglobulin and complement factors in  $\upbeta\cdot$ - thalassemia major patients in southern Iran. I journal immunology 4: 220-225.

24.Ahluwalia J, Datta U, Marwaha RK, Sehgal S (2000) Immune functions in splenectomised thalassemic children. Indian J Pediatr 67: 871-876.

25.Gharagozloo M, Karimi M, Amirghofran Z (2009) Double-faced cell- mediated immunity in beta-thalassemia major: stimulated phenotype versus suppressed activity. Ann Hematol 88: 21-27.

26.Javad G, Saeid A, Mohammadmehdi N (2011) Thalassemia and immune system dysfunction-review article. Int J Curr Res 3: 105-108.

27.Brand A (2002) Immunological aspects of blood transfusions. Transplant Immunology 10: 183-190.

28.Abbas A, Andrew H, Pilai S (2007) Cellular and molecular immunology. Philadelphia 2007 PA 19103-2899 SECTION II Recognion of An gens 77-92.

29.El Yazji MS (2011) Immunological assessment of beta-thalassemia major children aged 5-12 years old attending Abd El-Aziz El-Rantisy Hospital in Gaza strip. TURK J IMMUNOL 16: 1.

30.Aleem A, Shakoor Z, Alsaleh K, Algahtani F, Iqbal Z, et al. (2014) Immunological evaluation of  $\upbeta$ -thalassemia major patients receiving deferasirox. Journal of the College of Physicians and Surgeons Pakistan 24: 467-471.

31.Kiani-Amin M, Daneshi M, Ayazi P, Mohammadian S, Rezaei N (2011) Serum immunoglobulin levels in splenectomized and non- splenectomized patients with major Beta-thalassemia. Iran J Pediatr 21: 95-98.

32.Waghorn DJ (2001) Overwhelming infection in asplenic patients: current best practice preventive measures are not being followed. J Clin Pathol 54: 214-218.

33.Moshtaghi-kashanian GR, Gholamhoseinian A, Hoseinimoghadam A, Rajabalian S (2006) Splenectomy changes the pattern of cytokine production in beta-thalassemic patients. Cytokine 35: 253-257.

34.Shete A, Thakar M, Abraham PR, Paranjape R (2010) A review on peripheral blood  $\mathrm{CD4+\ensuremath{\,\mathrm{~T~}\,}~}$  lymphocyte counts in healthy adult Indians. Indian J Med Res 132: 667-675.

35.Bains I, Antia R, Callard R,Yates AJ (2009) Quantifying the development of the peripheral naive  $\mathrm{CD4+}$ T-cell pool in humans. Blood 113: 5480-5487.

36.Kadam PP, Manglani MV, Sharma SM, Sharma RA, Setia MS (2014) Immunoglobulin Levels and CD4 / CD8 Counts in Thalassemia Major. Indian pediatric 8: 1000-1002.

37.Noulsri E, Lerdwana S, Fucharden S, Pattanapanyasat K (2014) Phenotypic characterization of circulating CD4/CD8 T-lymphocytes in beta thalassemia patients. Asian Pac J Allergy Immunol 32: 261-269.

38.Del Vecchio GC, Schettini F, Piacente L, De Santis A, Giordano P, (2002) Effects of deferiprone on immune status and cytokine pattern in thalassaemia major. Acta Haematologica 108: 144-149.

39.Lee AC, Li CH (2004) Age as a factor in severe bacterial infection in transfusion-dependent patients with thalassemia major. Clin InfectDis 38: 1194-1195.

40.Pattanapanyasat K, Thepthai C, Lam-chiagdhase P, Lerdwana S, Tachava- nich K, et al. (2000) Lymphocyte subsets and specific T-cell immune response in thalassemia. Cytometry 42: 11-17.

41.Cunningham-Rundles S, Giardina PJ, Grady RW, Califano C, McKenzie P, et al. (2000) Effects of transfusional iron overload on immuneresponse. The Journal of Infectious Diseases 182: S115-S121.

42.Consolini R, Calleri A, Legitimo A, Massei F (2001) Immunological evaluation of patients with beta-thalassemia major. Acta Haematol 105: 7-12.

43.Ahmadiafshar A, Ghadipasha A, Mousavinasab N (2012) Immunological evaluation in patients with beta thalassemia major. Arch Dis Child 97: A217.

44.AA Hagag, A Mohamed, ES Elgamasy, A Elbar (2015) Assessment of T lymphocyte subsets in children with beta thalassemia major with iron overload. Egypt J Pediatr Allergy Immunol 13: 57-63.

45.Li  $\mathrm{K},$ Li CK, Wong RP, Wong A, Shing MM, et al. (1997) Transfusion related immunomodulation in Chinese children with thalassemia. Vox Sang 73: 167-173.

46.Speer CP, Gahr M, Schuff-Werner P, Schroter W (1990) Immunologic evaluation of children with homozygous beta-thalassaemia treated with desferrioxamine. ActaHaematol 83: 76-81.

47.Guglielmo P, Cunsolo F, Lombardo T, Sortin G, Giustolisi R, et al. (1984) T-subset abnormalities in thalassaemia intermedia: Possible evidence for a thymus functional deficiency. Acta Haematol 72: 361-367.

48.Singer S, Wu V, Mignacca R (2000) Alloimmunization and erythrocyte autoimmunization in transfusion-dependent thalassemia patients of predominantly. Asian Descent. Blood 96: 3369-3373.

49.Rachmilewitz EA, Giardina PJ (2011) How I treat thalassemia. Blood 118: 3479-3488.
A Compact Loop Type Antenna for Bluetooth, S-DMB, Wibro, WiMax, and WLAN Applications

Yong-sun Shin and Seong-Ook Park, Member, IEEE

Abstract—This paper presents a novel compact loop type an- tenna for Bluetooth, satellite-digital multimedia broadcasting, wireless broadband, and wireless local-area network applications. The proposed antenna fed by a  $\mathbf{50}\mathbf{-}\pmb{\Omega}$  coaxial line occupies a compact volume of  $\mathbf{26\,\times\,8\,\times\,3\,\,mm^{3}}$ 83 mm. The bandwidth is  $\mathbf{19.9\%}$   $(2.26{-}2.76~\mathbf{GHz})$  and  $45.6\%$  (4.03–6.4 GHz) at  $\mathbf{S_{11}}\,<\,-\mathbf{10}~\mathbf{dB}$ 10 dB, and the measured maximum gain is 6.41 dBi at 5.35 GHz. The Microwave Studio simulation is employed for optimizing the design parameters, and the simulated results agree well agree with those of the measured one.

Index Terms—Bluetooth, broadband, compact, loop antenna, multiband, satellite-digital multimedia broadcasting (S-DMB), wireless broadband (Wibro), wireless local-area network (WLAN), world interoperability for microwave access (WiMax).

I. INTRODUCTION A S modern mobile communication systems have developed rapidly, the antenna, which can cover multiband and have broadband, is needed for mobile handsets. In addition, for next- generationmobilehandsets,theantennahastobecompact,lower proﬁle, light, and easy to be embedded because the size of the an- tennaoftenhasagreatinﬂuenceonthewholesizeofwirelesssys- tems. There is generally the tradeoff between the size and perfor- mance of antenna because the characteristics of the antenna are closely concerned with its size. In the case of a planar inverted-F antenna, which is attractive to apply mobile handsets, the height of antennas is the critical factor for obtaining good performances [1]–[3]. Accordingly, it is not easy to realize the compact struc- ture. However, in the case of loop antennas, the antenna dimen- sions can be made sufﬁciently small and compact to be placed in the limited space provided on the equipment. Moreover, good performances required can be obtained [4]–[8].

In this paper, a novel compact loop type antenna is intro- duced for Bluetooth, satellite digital multimedia broadcasting (S-DMB), wireless broadband (Wibro in Korea), world interop- erability for microwave access (WiMax), and wireless local-area network (WLAN) applications. The size of the proposed an- tenna is  $26\times8~\times~3\,\mathrm{{mm}^{3}}$ 83mm with a circumference is approximately  $64.7~\mathrm{mm}$  corresponding to a half-wavelength at  $2.3~\mathrm{GHz}$ . For reducing the height and obtaining broad bandwidth, slots are applied in the feed part of the proposed antenna. Moreover, for good impedance matching, the distance between the feeding and shorting plate and the width of the lower plate of the antenna are changed. The bandwidths of the antenna atdB can be achieved about  $500\ \mathrm{MHz}$  ( $2.26{-}2.76~\mathrm{GHz})$  and  $2370~\mathrm{MHz}$   $(4.03{-}6.4~\mathrm{\GHz})$  suitable for Bluetooth  $(2.4{-}2.4835~\mathrm{\GHz})$ , S-DMB  $(2.63{-}2.655\,\mathrm{GHz})$ ), Wibro  $(2.3{-}2.39$  in Korea), WiMax  $(2.5{-}2.7\;\mathrm{GHz})$ , and WLAN (5.15–5.35,  $5.725–5.875\;\mathrm{GHz})$  ap- plications. This letter presents the experimental results of the impedance bandwidth and radiation characteristics compared to simulations.



metal is only printed on the front surface of an FR-4 substrate with relative permittivity 4.6. As seen in Fig. 1(a), the proposed loop type antenna is fed by a  $50–\Omega$ coaxial line and the inner conductor of the coaxial line is directly connected to the feed part in the ground plane.

The proposed antenna is based on a half-wavelength modiﬁed square loop structure with a size of  $26\times8~\times~3\:\bar{\mathrm{mm^{3}}}$ 83 mm and the depth of the copper strip is  $0.2\,\mathrm{mm}$ . The effective circumference of the antenna is approximately  $64.7~\mathrm{mm}$  corresponding to a half-wavelength at  $2.3~\mathrm{GHz}$  as shown in Fig. 1(b). The major tuning parameters are slots near the feed part of the antenna, $d$ , , and. For obtaining low proﬁle structure and multiband, there are slots near the feed part of the antenna, and the step shape is in the bent position.

 



Moreover, the upper and lower metals have different widths. In order to obtain optimum antenna performances, Microwave Studio (MWS) is used. Fig. 2 shows simulated results depending on with and without slot in the feed part of the proposed antenna, which adjusts the distance between feeding and shorting plate and the width of the lower metal. As seen in Fig. 2(a) and (b), there are good impedance matching and broad bandwidth with slots in feed part of the antenna and according to the distance be- tween feed line and shorting plate. Moreover, from Fig. 2(c), the wider the lower metal width, the better the impedance matching. The proposed loop type antenna is manufactured and measured, and experimental results are presented and compared to simu- lated results in the following section.

measured with an Anritsu 37377C network analyzer. Fig. 3 shows the measured and simulated scattering parameters with  $d=2.3\:\mathrm{mm}$ , $w_{1}=2\;\mathrm{mm}$ , and $w_{2}\,=\,3.5\ \mathrm{mm}$ , respectively. As shown in Fig. 3, the experimental data generally agree



TABLE I THE MEASURED GAINS ACCORDING TO FREQUENCIES

with the simulated results. The measured impedance band- width atdB is about  $500~\mathrm{MHz}$  ( $2.26{-}2.76~\mathrm{GHz})$  and 2370 MHz  $(4.03{-}6.4~\mathrm{GHz})$ ) covering Bluetooth, S-DMB, Wibro, WiMax, and WLAN bands. In order to check the cur- rent ﬂows at the resonant frequencies, 2.43 and 5.91 GHz, the simulation results in Fig. 4 have been obtained from MWS. As seen in Fig. 4, these total lengths of the current paths are about 32 and  $12~\mathrm{mm}$ , which are corresponding to an approximately quarter-wavelength at 2.43 and 5.91 GHz, respectively. The radiation characteristics of the proposed antenna also have been investigated. Figs. 5 and 6 show the compared results of the measured and simulated radiation patterns at 2.43 and 5.91 GHz. There are in good agreement with each other. Table I shows the measured gains according to frequencies. The measured maximum gain of the proposed loop type antenna is 6.41 dBi at  $5.35~\mathrm{GHz}$ .

IV. CONCLUSION A novel loop type antenna for Bluetooth, S-DMB, Wibro, and WLAN applications has been introduced and experimentally in- vestigated. The proposed antenna has a compact dimension of  $26\times{\bar{8}}\;\times\;3\;\mathrm{{mm}^{3}}$ 83 mm and has obtained the broad bandwidth by ad- justing tuning parameters $d,w_{1},w_{2}$ ,. The bandwidth of the pro- posed antenna with $d=2.3\,\mathrm{mm}$ , $w_{1}=2\:\mathrm{mm}$ , $w_{2}=3.5\:\mathrm{mm}$  is  $19.9\%$   $(2.26{-}2.76~\mathrm{GHz})$  and  $45.6\%$   $(4.03{-}6.4~\mathrm{GHz})$  covering the Bluetooth  $(2.4{-}2.4835~\mathrm{GHz})$ ), S-DMB  $(2.63{-}2.655~\mathrm{GHz})$ , Wibro (2.3–2.39 in Korea), WiMax  $(2.5{-}2.7\,\mathrm{GHz})$ , and WLAN  $(5.15{-}5.35,\,5.725{-}5.875\ \mathrm{GHz})$  bands at $\mathrm{S_{11}\,<\,-10~d B}$ . The radiation characteristics were also observed, and the measured maximum gain is 6.41 dBi at  $5.35\;\mathrm{GHz}$ .

 



REFERENCES [1] C. R. Rowell and R. D. Murch, “A capacitively loaded PIFA for com- pact mobile telephone handsets,” IEEE Trans. Antennas Propag.



45, no. 5, pp. 837–842, 1997.

[2] T. Tanaka, S. Hayashida, K. Imamura, H. Mrishita, and Y. Koyanagi, “A study on miniaturization of a handset antenna utilizing magnetic materials,” in Proc. 10th Asia-Paciﬁc Conf. Commun. 5th Int. Symp. Multi-Dimensional Mobile Commun., 2004, vol. 2, pp. 665–669.

[3] P. Nepa, G. Manara, A. A. Serra, and G. Nenna, “Multiband PIFA for WLAN mobile terminals,” IEEE Antennas Wireless Propag. Lett., vol. 4, pp. 349–350, 2005.

[4] C. C. Lin, G. Y. Lee, and K. L. Wong, “Surface-mount dual-loop an- tenna for 2.4/5 GHz WLAN operation,” Electron. Lett., vol. 39, no. 18, pp. 1302–1303, 2003.

[5] B. Bai and Z. H. Feng, “A dual band mandering folded loop antenna,” in Proc. IEEE Antennas Propag. Soc. Int. Symp., 2004, vol. 4, pp. 20–25. [6] S. Tanaka, S. Hayashida, H. Morishita, and Y. Atsumi, “Wideband and compact folded loop antenna,” Electron. Lett., vol. 41, no. 17, pp. 945–946, 2005.

[7] K. Subramaniam, M. Esa, and S. Subahir, “Compact printed square meander loop antenna for a Bluetooth headset,” in Proc. 2005 Asia- Paciﬁc Conf. Appl. Electromagn., 2005, pp. 86–90.

[8] S. Blanch, J. Romeu, and I. Corbella, “Wide-band high-efﬁciency printed loop antenna design for wireless communication systems,” IEEE Trans. Veh. Technol., vol. 54, no. 3, pp. 873–878, 2005.
A Compact Loop Type Antenna for Bluetooth, S-DMB, Wibro, WiMax, and WLAN Applications

Yong-sun Shin and Seong-Ook Park, Member, IEEE

Abstract—This paper presents a novel compact loop type an- tenna for Bluetooth, satellite-digital multimedia broadcasting, wireless broadband, and wireless local-area network applications. The proposed antenna fed by a  $\mathbf{50}\mathbf{-}\pmb{\Omega}$  coaxial line occupies a compact volume of  $\mathbf{26\,\times\,8\,\times\,3\,\,mm^{3}}$ 83 mm. The bandwidth is  $\mathbf{19.9\%}$   $(2.26{-}2.76~\mathbf{GHz})$  and  $45.6\%$  (4.03–6.4 GHz) at  $\mathbf{S_{11}}\,<\,-\mathbf{10}~\mathbf{dB}$ 10 dB, and the measured maximum gain is 6.41 dBi at 5.35 GHz. The Microwave Studio simulation is employed for optimizing the design parameters, and the simulated results agree well agree with those of the measured one.

Index Terms—Bluetooth, broadband, compact, loop antenna, multiband, satellite-digital multimedia broadcasting (S-DMB), wireless broadband (Wibro), wireless local-area network (WLAN), world interoperability for microwave access (WiMax).

I. INTRODUCTION A S modern mobile communication systems have developed rapidly, the antenna, which can cover multiband and have broadband, is needed for mobile handsets. In addition, for next- generationmobilehandsets,theantennahastobecompact,lower proﬁle, light, and easy to be embedded because the size of the an- tennaoftenhasagreatinﬂuenceonthewholesizeofwirelesssys- tems. There is generally the tradeoff between the size and perfor- mance of antenna because the characteristics of the antenna are closely concerned with its size. In the case of a planar inverted-F antenna, which is attractive to apply mobile handsets, the height of antennas is the critical factor for obtaining good performances [1]–[3]. Accordingly, it is not easy to realize the compact struc- ture. However, in the case of loop antennas, the antenna dimen- sions can be made sufﬁciently small and compact to be placed in the limited space provided on the equipment. Moreover, good performances required can be obtained [4]–[8].

In this paper, a novel compact loop type antenna is intro- duced for Bluetooth, satellite digital multimedia broadcasting (S-DMB), wireless broadband (Wibro in Korea), world interop- erability for microwave access (WiMax), and wireless local-area network (WLAN) applications. The size of the proposed an- tenna is  $26\times8~\times~3\,\mathrm{{mm}^{3}}$ 83mm with a circumference is approximately  $64.7~\mathrm{mm}$  corresponding to a half-wavelength at  $2.3~\mathrm{GHz}$ . For reducing the height and obtaining broad bandwidth, slots are applied in the feed part of the proposed antenna. Moreover, for good impedance matching, the distance between the feeding and shorting plate and the width of the lower plate of the antenna are changed. The bandwidths of the antenna atdB can be achieved about  $500\ \mathrm{MHz}$  ( $2.26{-}2.76~\mathrm{GHz})$  and  $2370~\mathrm{MHz}$   $(4.03{-}6.4~\mathrm{\GHz})$  suitable for Bluetooth  $(2.4{-}2.4835~\mathrm{\GHz})$ , S-DMB  $(2.63{-}2.655\,\mathrm{GHz})$ ), Wibro  $(2.3{-}2.39$  in Korea), WiMax  $(2.5{-}2.7\;\mathrm{GHz})$ , and WLAN (5.15–5.35,  $5.725–5.875\;\mathrm{GHz})$  ap- plications. This letter presents the experimental results of the impedance bandwidth and radiation characteristics compared to simulations.



metal is only printed on the front surface of an FR-4 substrate with relative permittivity 4.6. As seen in Fig. 1(a), the proposed loop type antenna is fed by a  $50–\Omega$ coaxial line and the inner conductor of the coaxial line is directly connected to the feed part in the ground plane.

The proposed antenna is based on a half-wavelength modiﬁed square loop structure with a size of  $26\times8~\times~3\:\bar{\mathrm{mm^{3}}}$ 83 mm and the depth of the copper strip is  $0.2\,\mathrm{mm}$ . The effective circumference of the antenna is approximately  $64.7~\mathrm{mm}$  corresponding to a half-wavelength at  $2.3~\mathrm{GHz}$  as shown in Fig. 1(b). The major tuning parameters are slots near the feed part of the antenna, $d$ , , and. For obtaining low proﬁle structure and multiband, there are slots near the feed part of the antenna, and the step shape is in the bent position.

 



Moreover, the upper and lower metals have different widths. In order to obtain optimum antenna performances, Microwave Studio (MWS) is used. Fig. 2 shows simulated results depending on with and without slot in the feed part of the proposed antenna, which adjusts the distance between feeding and shorting plate and the width of the lower metal. As seen in Fig. 2(a) and (b), there are good impedance matching and broad bandwidth with slots in feed part of the antenna and according to the distance be- tween feed line and shorting plate. Moreover, from Fig. 2(c), the wider the lower metal width, the better the impedance matching. The proposed loop type antenna is manufactured and measured, and experimental results are presented and compared to simu- lated results in the following section.

measured with an Anritsu 37377C network analyzer. Fig. 3 shows the measured and simulated scattering parameters with  $d=2.3\:\mathrm{mm}$ , $w_{1}=2\;\mathrm{mm}$ , and $w_{2}\,=\,3.5\ \mathrm{mm}$ , respectively. As shown in Fig. 3, the experimental data generally agree



TABLE I THE MEASURED GAINS ACCORDING TO FREQUENCIES

with the simulated results. The measured impedance band- width atdB is about  $500~\mathrm{MHz}$  ( $2.26{-}2.76~\mathrm{GHz})$  and 2370 MHz  $(4.03{-}6.4~\mathrm{GHz})$ ) covering Bluetooth, S-DMB, Wibro, WiMax, and WLAN bands. In order to check the cur- rent ﬂows at the resonant frequencies, 2.43 and 5.91 GHz, the simulation results in Fig. 4 have been obtained from MWS. As seen in Fig. 4, these total lengths of the current paths are about 32 and  $12~\mathrm{mm}$ , which are corresponding to an approximately quarter-wavelength at 2.43 and 5.91 GHz, respectively. The radiation characteristics of the proposed antenna also have been investigated. Figs. 5 and 6 show the compared results of the measured and simulated radiation patterns at 2.43 and 5.91 GHz. There are in good agreement with each other. Table I shows the measured gains according to frequencies. The measured maximum gain of the proposed loop type antenna is 6.41 dBi at  $5.35~\mathrm{GHz}$ .

IV. CONCLUSION A novel loop type antenna for Bluetooth, S-DMB, Wibro, and WLAN applications has been introduced and experimentally in- vestigated. The proposed antenna has a compact dimension of  $26\times{\bar{8}}\;\times\;3\;\mathrm{{mm}^{3}}$ 83 mm and has obtained the broad bandwidth by ad- justing tuning parameters $d,w_{1},w_{2}$ ,. The bandwidth of the pro- posed antenna with $d=2.3\,\mathrm{mm}$ , $w_{1}=2\:\mathrm{mm}$ , $w_{2}=3.5\:\mathrm{mm}$  is  $19.9\%$   $(2.26{-}2.76~\mathrm{GHz})$  and  $45.6\%$   $(4.03{-}6.4~\mathrm{GHz})$  covering the Bluetooth  $(2.4{-}2.4835~\mathrm{GHz})$ ), S-DMB  $(2.63{-}2.655~\mathrm{GHz})$ , Wibro (2.3–2.39 in Korea), WiMax  $(2.5{-}2.7\,\mathrm{GHz})$ , and WLAN  $(5.15{-}5.35,\,5.725{-}5.875\ \mathrm{GHz})$  bands at $\mathrm{S_{11}\,<\,-10~d B}$ . The radiation characteristics were also observed, and the measured maximum gain is 6.41 dBi at  $5.35\;\mathrm{GHz}$ .

 



REFERENCES [1] C. R. Rowell and R. D. Murch, “A capacitively loaded PIFA for com- pact mobile telephone handsets,” IEEE Trans. Antennas Propag.



45, no. 5, pp. 837–842, 1997.

[2] T. Tanaka, S. Hayashida, K. Imamura, H. Mrishita, and Y. Koyanagi, “A study on miniaturization of a handset antenna utilizing magnetic materials,” in Proc. 10th Asia-Paciﬁc Conf. Commun. 5th Int. Symp. Multi-Dimensional Mobile Commun., 2004, vol. 2, pp. 665–669.

[3] P. Nepa, G. Manara, A. A. Serra, and G. Nenna, “Multiband PIFA for WLAN mobile terminals,” IEEE Antennas Wireless Propag. Lett., vol. 4, pp. 349–350, 2005.

[4] C. C. Lin, G. Y. Lee, and K. L. Wong, “Surface-mount dual-loop an- tenna for 2.4/5 GHz WLAN operation,” Electron. Lett., vol. 39, no. 18, pp. 1302–1303, 2003.

[5] B. Bai and Z. H. Feng, “A dual band mandering folded loop antenna,” in Proc. IEEE Antennas Propag. Soc. Int. Symp., 2004, vol. 4, pp. 20–25. [6] S. Tanaka, S. Hayashida, H. Morishita, and Y. Atsumi, “Wideband and compact folded loop antenna,” Electron. Lett., vol. 41, no. 17, pp. 945–946, 2005.

[7] K. Subramaniam, M. Esa, and S. Subahir, “Compact printed square meander loop antenna for a Bluetooth headset,” in Proc. 2005 Asia- Paciﬁc Conf. Appl. Electromagn., 2005, pp. 86–90.

[8] S. Blanch, J. Romeu, and I. Corbella, “Wide-band high-efﬁciency printed loop antenna design for wireless communication systems,” IEEE Trans. Veh. Technol., vol. 54, no. 3, pp. 873–878, 2005.
# Benchmarks for a History   of Liturgical Formation

## PATRICk PRéTOT, OSB\*

A s I begin, I would like to extend my warmest thanks to the council and those  who organized the Québec Congress for having invited me to speak today. I  was particularly touched by this invitation because, as you know, I had to stand  aside, much to my regret, from the organization of this Congress. So I would like  to express before you all my grateful affection towards Lizette Larson-Miller, for  this gesture of fellowship, and above all for having accepted to take over from me  as president. Our president elect Martin Stuflesser and all the members of the  council are of course included in this heartfelt thank-you. To be perfectly truthful,  I should mention that I hesitated before accepting this invitation to speak here in  Québec, because the underlying reasons for my well-founded decision not to come  to Würzburg remain. At the same time, I had to re-orient my future academic  obligations. And so it is with modesty and gratitude that I stand here before you  this morning. I am as aware of the extent of the honour that you bestow on me as  of my limits as to the task at hand.

### InTroducTIon

This presentation is obviously not going to attempt a synthesis of the history of  liturgical formation. The subject is not a totally unexplored area, but as far as my  research into the question has been able to enlighten me, it remains a subject that  has been not much studied for itself, even by liturgical historians. When the  question has indeed been taken up, it has been principally approached from the  point of view of contemporary problematics, and in the context of reflection upon  liturgical theology and pastoral practice.



We should also take note that in standard French studies of the history of education,  for example Henri-Irénée Marrou’s L’histoire de l’éducation dans l’Antiquité5 or  Pierre Riché’s (Marrou’s disciple and successor at the Sorbonne) classic works,  Éducation et culture dans l’Occident barbare and Ecoles et enseignement dans  $l e$   Haut Moyen  ${\hat{A}}g e^{6},$  the question of “liturgical formation” is not dealt with as such.  It does make an appearance in these authors’ studies of Christian Initiation7, but  the question of how laity and clergy were formed to liturgical life is not raised.  This relative silence is no doubt the result of separation between different fields of  academic study; although historians of liturgy and historians of education are not  unaware of each other’s existence, the dynamics of their respective studies have  never led them to construct a common object for their research.



Although the subject of liturgical history does not appear in the tables of contents  of historical publications, the question itself is far from absent. For example, Pierre  Riché, in a passage on “Elementary and Specialised Teaching”, points out that in  early medieval programmes of study, “reading and chant were complementary  subjects. Students memorized through song not only psalms, but also the short  Latin texts they were to study”. Then he adds this important remark: All children, even if they do not have good voices, were to learn to sing. When  Charlemagne made cantus part of his programme’s obligatory core content, his  object was both general formation and liturgical studies. The children were to  start participating in liturgical services, and to learn to sing on the basis of  psalms, hymns and canticles.



So it is already clear that, from the point of view of history, at least in the earlier  periods, liturgical formation is probably “hiding itself” in “general formation,” and  runs the risk of remaining invisible as much to historians as to liturgists. We may  even make the hypothesis that, over time, liturgical formation will become all the  more visible and necessary in so far as “general formation” loses its living link to  the liturgy. This first element can orient further research; at a time when processes  of transmission are breaking down, it would be helpful to dig deeper into this  fundamental aspect, in particular on the basis of documents like hagiographical  sources and canonical collections, with a view to coming to a better understanding  of the way the major benchmarks of liturgical experience were handed on in the  past, within Christian living. 



Having set this background, I shall now seek to shed light on the relationship played  out through history between, on one hand, liturgy in its celebration, and on the  other, what we can place under the approximate heading of “liturgical formation”. I  have just used the adjective “approximate” because it is clear to me that the notion  of “liturgical formation” hides a multitude of realities. It is to be hoped that the  different approaches that will become apparent during our Congress will help us  to delineate the subject with greater clarity, a subject whose foundational character  was underlined by the Liturgical Movement, and whose essential importance for  teachers of liturgy is now evident. The risk we run is that of a certain confusion if  we do not distinguish the varied realities lurking behind this notion.

At the least, a historical survey can help us to realize just how complex the idea  of “liturgical formation” is, by calling into question oversimplifications which  present it as a homogenous reality over the course of time. The very diversity  of the expressions used to designate it though history is a sign of a constant  plurality of practices as well as of understandings. Today, under the influence of  the Liturgical Movement, we usually speak of “liturgical formation” or “formation  for liturgy”, and while the term “mystagogy” has recently reappeared, in the Early  Modern period liturgists seem to have favoured the expressions “explanations”  and “instructions”.

In order to set the character of liturgical formation into its historical context, or  rather contexts, I shall divide my account into historical stages, with a view to  identifying the various actors, places and moments that play a role in the process, as  well as to working out as much as possible of its aims and methods. This approach  is not without risk. There will be no difficulty to find exceptions that do not fit into  the suggested outline. Moreover, in order fully to take into account the complexity  of the historical reality, it would be necessary to extend the investigation well  beyond what is possible here. However, in the absence of a more precise survey,  this approach may at least allow us to reach a general idea of the question.

To begin, a few methodological reflections seem necessary.

  Voir par exemple, Nicolas Le Tourneux, Instructions chrétiennes sur les sacrements et sur  les cérémonies avec lesquelles l’Eglise les administre, Paris, Hélie Josset, 1686 ; Jean-Jacques Ollier,  Explication des cérémonies de la grand’messe messe de paroisse selon l’usage romain, Paris, Langlois, 1687  ; et encore Louis-Albert Joly De Choin, Instructions sur le rituel, concernant la théorie et la pratique des  sacrements et de la morale, et tous les principes et décisions nécessaires aux curés, confesseurs, prédicateurs,  chanoines, bénéficiers, prêtres, ou simples clercs, Lyon, chez les frères Perisse Imprimeurs Libraires, 1778  (nombreuses rééditions).

### 1.    on The noTIon oF “LITurgIcaL FormaTIon”

We should begin by taking note that the notion of “liturgical formation”, meaning  the transmission of an intellectual knowledge concerning liturgy, is no doubt a  recent idea, born in the Liturgical Movement, where it became one of its most vital  aspects. It is not going too far to say that this concern for formation was at the  origin of the development between 1841 and 1962 of an educative project that today  seems almost inconceivable. Furthermore, this project reached the whole of the  Christian people, and was embodied in numerous publications, articles, sessions,  congresses, and above all in the publishing of missals like those of Dom Gaspar  Lefebvre (1880–1966), a Benedictine of the monastery of Saint-André-lez-Bruges13.

This provisional definition of “liturgical formation” as “the transmission of an  intellectual knowledge concerning liturgy” matches what comes to mind naturally  when we hear the expression. Of course liturgical formation goes well beyond this  approximate and provisional definition; hence two further important remarks are  called for.

In the first place, just as the pairs “worship” – “liturgy” in English, “Liturgie” –   in German, and “ $^{\alpha}K u l t^{\gamma}$ liturgie” – “culte” in French are not perfectly equivalent,  the term “formation” probably does not point to the same mental representations  in our different languages and cultures. This is partly because of differing traditions  as to the manner of teaching. But it is also rooted in the very nature of the matter  at hand. What is at issue here is not only a “knowledge about” (savoir sur) liturgy,  but also “knowing how to be” (savoir-être), a “knowing how to do” (savoir-faire),  and even an important element of “knowing how to live” (savoir-vivre). So we can  already see that the notion of formation masks a highly complex reality. It is thus  probable that the relative silence on this subject that I pointed out earlier results  from the difficulty it presents.



In the second place, this is true not only in space, but also over time. While it is  certainly true that the Liturgical Movement had a decisive impact on the idea of  formation, it is important to counterbalance this by pointing out that a concern  for liturgical formation is by no means limited to the contemporary period. What  is true of liturgical inculturation is also true of liturgical formation; the Church  has always been confronted with the necessity of transmitting its practices,  even though the ways of so doing have differed over the course of time. This  preoccupation existed long before the Liturgical Movement, but no doubt in a  different manner, and, above all, without the use of the word “formation”.

The importance of this preoccupation is striking; the learned publications on the  liturgy published in France throughout the seventeenth and eighteenth centuries  favour the word “instruction” and employ an explanatory paradigm. It is no secret  that Early Modernity bears the mark of the forces unleashed by Luther and the  other reformers, and of their desire that the disciples of Christ not be abandoned  to ignorance. This desire was echoed within the Roman Catholic Church in the  educational institutions developed after the Council of Trent; here in Québec we  can cite the work of Early Modern religious orders like the Society of Jesus and  the Ursulines. Liturgical historiography over the past century has perhaps overly  neglected this Early Modern period because of the justifiable attraction for the  primitive and patristic periods. But the Early Modern period has much to teach us  about the prehistory of that concern for formation that was so important for the  Liturgical Movement.

### 2.    a hIsTorIcaL seTTIng To PermIT a generaL overvIew 

As I said in my introduction, what follows is an overview divided into historical  periods with a view to identifying the principal stages of the history of liturgical  formation. But as Father Pierre-Marie Gy often pointed out, although division  into historical periods can be helpful, such divisions are always somewhat artificial  and so partly false. The divisions created never quite match the slow evolution  of practices and representations that characterize liturgical life. However, and  despite these necessary precautions, we shall adopt a division into four or five  main periods. The first period runs until the emergence of the earliest liturgical  institutions; hence it ends towards the middle of the fourth century. The second  period is the “high” patristic era. The third stage is the Middle Ages. It is important  here to bear in mind that the Middle Ages is anything but a homogenous block,  and with regard to formation, the fourteenth and fifteenth centuries are worlds  away from the ninth century; the emergence of universities in the thirteenth  century constitutes a sort of turning point.

Finally, the Early Modern / Modern period brings with it a new situation. The  Liturgical Movement was in part the product of this situation, and at the same  time the result of a sustained effort to move on, as can be seen in certain of Dom  Guéranger’s writings. So it would be possible to speak of a fifth period inaugurated  by the Liturgical Movement, because of the ways it transformed participation in  the liturgy as well as its methods of scientific investigation that would serve as a  basis for formation. Liturgical formation is durably marked by the development  of historical studies of liturgy following on from Msgr Duchesne. Indeed over  several decades liturgical formation was more or less identified with a knowledge  of liturgical history.



Within the setting of a paper delivered to this Congress, without going into detail,  I shall limit myself to calling attention to the essential structures of formation,  by identifying its principal actors, the context of the act of formation, and the  contents that were transmitted.

I am indebted for my hypothesis to an article on “practical and spiritual theology” by  René Marlé S.J., in which he suggests understanding the diversification of theology  over time as a process of progressive distinction and even separation between  Scripture, theology, liturgy and spirituality. In this evocative synthesis written  for a theology textbook, Marlé notes that in the patristic age, as well as during  part of the Middle Ages (the example of St. Anselm comes to mind), the form of  theological discourse was “above all, if not exclusively, a scriptural commentary.”  He adds that in this situation, “theology was directly spiritual, indeed it was also  pastoral, and developed spontaneously into prayer or exhortation.” But he goes on  to point out that since the later Middle Ages theology has taken: … the form of an often highly rarefied and subtle critique of the language of  faith and of the possibilities of discourse concerning God, and … in parallel,  a growing number of works and pamphlets, seeking, in varying registers, to  account for the experience of faith.



In other terms, since the Middle Ages, the splitting of theological reflection into  Scripture, theology and liturgy has isolated liturgy as a ritual practice generally  safeguarded by a body of rules destined to ensure its correct execution. This  splitting can be said to have “given sanctuary” to liturgy by cutting it off from  ethical responsibility and even, all too often, from the seeking of God. My  intention is now to consider our different historical periods from the point of view  of liturgical formation in the light of this hypothesis.



### 3.    The PrImITIve church

After the first generations follows the period of the apostolic Fathers, a time that  can be described as “pre-Constantinian”, covering the second and third centuries.  I shall group these two periods together, although an attentive reading of our past  president Gordon Lathrop’s recent book, The Four Gospels on Sunday, would invite  us to distinguish these two periods; the years during which the biblical canon  was slowly worked out obeyed a dynamic that escapes us for the most part, but  which also certainly left an essential mark on the formation of believers.

During these early periods, Christian institutions were still in gestation; their  embryonic forms were tributary to their regional contexts, and above all handed  on orally. Only a tiny quantity of evidence remains, which raises the permanent  risk of reading too much into it. We can of course cite important documents like  the Didachè, the letters of Ignatius of Antioch, Tertullian’s work on baptism,  or even, and especially, the Apostolic Tradition. Nonetheless, it is now widely  recognized how quickly any attempt to suggest general reconstructions runs up  against inevitable historiographical difficulties. However, it is probably true that  in this early period, the inheritance of Jewish institutions played a significant role,  so that the study of the Jewish origins of Christian worship may be illuminating.  But it is also clear that recent historiography on this subject lacks sufficient  consensus to draw firm conclusions.



During this first period, the first actors were without doubt the apostles, but  also, after them, and before the hierarchical structure: bishop/presbyterium and  deacons, emerged, didascales carried the new faith’s message. So it was the Gospel,  that is to say the proclamation of the Good News, in particular through the  Gospel stories, but also Paul’s letters, that constituted the first means of liturgical  formation. As Gordon Lathrop stressed in his opening address at the Würzburg  Congress, these were concerned not so much with saying something about  liturgy as with reforming assemblies so that they might live as authentic disciples  of Jesus. Of course, while the New Testament and the earliest Christian writings  were never textbooks for liturgical formation, we may still rapidly take note of two  examples: the first Letter of Paul to the Corinthians (1 Cor. 11:17-34) and Justin’s  First Apology.



When the apostle Paul wrote to believers in Corinth, we know that he undertook  a stern critique of their assemblies. He began by pointing out that these assemblies  “did more harm than good” (11:17), he did not hesitate to denounce the divisions  of the community (11:18-19), going as far as to discredit meetings that were not  what they were supposed to be; for he said that “when you all come together, it is  no longer the Lord’s supper that you take” (11:20).

At a later date, this passage would be too often interpreted in the light of verse 27:  “the one who will have eaten the bread or drunk the cup of the Lord unworthily  will have to answer for the body and blood of the Lord”, focalizing discernment  on the conscience of the individual who receives communion, an insistence which  has often kept believers away from participation in the eucharistic table. In  reality, Paul is recalling a received tradition (11: 23b-25) as an invitation to the  Corinthians not to pervert what is given in order to be lived in the Lord’s supper,  that is to say, “whenever you eat this bread and drink this cup, you proclaim the  death of the Lord until he comes”.

This is certainly an act of formation, one whose object is the authenticity of practices.  It is not something “about” the eucharist, but rather it seeks to foster Christian life;  and the eucharist is the criterion of the truth of Christian life. So we have here  a clear indication of how inseparable liturgical formation and ethical formation  are at this point in history. It is unsettling to realize that over history, respect for  traditions or for ritual regulations has at times coexisted with the sort of deviations  of meaning that Paul did not hesitate to denounce. For the Corinthians were  no doubt acting in accord with what they themselves thought they had received,  and indeed Paul does not ask them to modify the ritual order of their action. The  history of the Church can doubtless provide us with numerous examples of what  we could call perversions of liturgy. It bears repeating that the perversion is not in  the first place deviation from the ritual observance but a subversion of the action  inasmuch as it realises the saving work carried out by Christ’s Passover.

For Justin, liturgical formation seems to be closely linked to a vision of the Church  as a “priestly people”. This is why the essential setting of formation is the Sunday  assembly, not so much as the coming together of those who have followed the Way  as the manifestation of the priestly community of believers. For in the assembly  that meets on the Lord’s Day resound the “memoirs of the apostles” and “the  writings of the prophets”, and “the one who presides” “lifts up to heaven prayers  and thanksgivings”, and are shared not only “consecrated foods”, but other goods  in aid of “all who are in need”.

Hence it is clear that in this earliest period liturgical formation is undertaken  by those who announce the Gospel, and its content can be identified with that  Tradition which Paul says he has received from the Lord himself. This formation  takes place especially within the Sunday assembly and so is addressed to the whole  of the Christian people.

### 4.    The “hIgh” PaTrIsTIc PerIod

Of course, more than the preceding period, the age that has often been presented  as the golden age of the liturgy as well as that of the formation of the Christian  people for the liturgy, is the first that comes to mind when liturgical formation is  mentioned. However, as Robert Taft reminds us, liturgists must beware of making  the patristic era (fourth to sixth centuries) a “golden age”, even if it certainly is  a fascinating time for our subject. The fourth century, that saw the major  institutions of the Christian liturgy take form, in particular the catechumenate  and the development of the liturgical year, can be seen as expressing a primordial  concern for Christian formation; the catecheses that accompanied the process of  Christian initiation show that this concern for Christian formation is inseparable  from liturgical formation. Directly concerning our subject, the patristic  sacramental catecheses have even sometimes appeared to be a sort of normative  model for liturgical formation. The proximity of catechesis and liturgy during this  period can indeed lead us to wonder whether ancient catechesis was not primarily  liturgical formation, that is to say formation “by” and  $^{\mathrm{{c}}}{\mathrm{{in}}}^{\mathrm{{\nu}}},$  or “within”, liturgy.



The complexity of this period invites us to guard against an idealisation of the  fruits that may have been borne by the mystagogical teaching of the likes of Cyril  of Jerusalem, John Chrysostom or Theodore of Mopsuestia in the East, or the  baptismal catecheses of Ambrose of Milan and Augustine of Hippo. Nonetheless,  these outstanding texts remain sources of inspiration for the liturgical formation  of believers. We can take note of the fact that in France the classic distinction  between pre-baptismal and post-baptismal catechesis, along with the idea and  practice of mystagogy (at least in part)46 have been recovered as a result of a  general effort of catechetical renewal. But here again, the emphasis placed on  certain types of sources runs the risk of making us forget that liturgical formation  during this period also took many other routes, in particular the type of preaching  exemplified by Leo the Great or Gregory the Great49, what we call today liturgical  catechesis. But even though these documents must be accorded an utmost  importance, many others should be taken into account, and perhaps first among  them the hagiographical narratives that were beginning to come into circulation,  through which the example of the saints transmitted a “knowing how to be”  (savoir-être) of Christian practices, including ritual practices.



We need to listen to Augustine here, for there are good reasons for holding him as  the master of Christian pedagogy. For example, in his discussion of the formation  of catechumens, the author of De Catechizandis rudibus points out: Concerning the sacrament they are to receive, it is enough for the more gifted  to be told the meaning of the rites, but with the less intelligent it is necessary to  provide further explanations and comparisons, so that they do not treat what  they see lightly.

 



Furthermore, we should remember that in the well-known passage concerning  Christian signs in book III of De Doctrina christiana, Augustine underlines  Christian liberty in this matter; he emphasises that in the economy of the New  Covenant, the multitude of signs that were in effect at the time “of the patriarchs,  the prophets, and all the figures of the history of Israel” has been replaced by  $^\mathrm{{\sc\alpha}}\mathrm{{\boldmath~a~}~}$  small number” “that the Lord himself and the teaching of the apostles has handed  on to us” (he is making reference to “baptism and the sacrament of the Body and  Blood of the Lord”) and which he says are “very easy to accomplish, very august  in their meaning, and very holy in their observance”. Indeed, for Augustine, “all  Christians, when they receive them, know, after being instructed, what they refer  to, and are thereby led to revere them, not out of fleshly servitude, but, on the  contrary, with spiritual freedom”.

Augustinian liturgical formation thus appears in all its pertinence; at a time  when the tendency to confuse the mechanical application of rules with a true ars  celebrandi reappears here and there, the vision of the great bishop of Hippo is more  important than ever: Now, – says Augustine – as to follow the letter and to take the signs for the  realities they symbolise comes from weakness and bondage, so to misinterpret  the signs comes from being misled by error.



It is clear that the patristic period was an important time for liturgical formation.  In continuity with the preceding period, the liturgical assembly is still the veritable  site of formation; the importance placed on due order in assemblies and episcopal  complaints about the absenteeism that is beginning to make itself felt testify to  this. The subject matter is too vast to develop here; however, we can sum up what  can be learned from this early period in two points.

First, as with mystagogical catecheses, it was not so much a matter of liturgical  formation as we understand it today as formation for Christian living. This  formation made use of Scripture as its primary source in order to hand on the  fundamental elements of the faith rooted in liturgical experience.

Second, this “formation” was given in the setting of the Christian assembly, normally  by bishops, or by priests or deacons in their name; there was thus an essential link  between liturgical formation and the service of the community. Liturgical formation  was not anyone’s specialty, but was part of the duties of all pastors.

### 5.    medIevaL LIghT and shadows

I had originally planned to spend more time on this particular period, because as  I prepared this lecture, my preliminary research led me to enriching discoveries  concerning the Middle Ages. As well, among historians, medievalists have paid  more attention than most to the question of the transmission of knowledge. But  also because it is particularly during this period that the concern with liturgical  formation begins to appear and find expression everywhere.

In conformity with this lecture’s general hypothesis, that of a splitting up of  Scripture, theology and spirituality beginning in the Middle Ages, the period  can be described as that of the veritable birth of formation “for” liturgy. Whereas  during the patristic period theology, and hence liturgical formation, took place  principally in the setting of the Christian assembly, and was thus an essentially  pastoral act, the gradual appearance of a specialised body of theologians would  little by little lead theological reflection to leave its initial community setting, and  to become the prerogative first of the masters of the monastic cloisters, then of the  masters of the university quadrangles.

Medieval writings on liturgy constitute an immensely rich body of works that took  form progressively from the De officiis of Isidore of Seville at the beginning of  the seventh century until the Rationale of Durandus of Mende (towards 1230-

1296) at the end of the thirteenth. We cannot do more here than evoke its scale,  without even being able to cite its principal elements. These liturgical summae,  a term we can use without fear of exaggeration, so clear is the intention of an  encyclopaedic presentation, are the counterparts of the theological summae of the  scholastic age. Formation for liturgy here takes the form of the transmission of  a rarefied knowledge of the rites and their meaning, a transmission which was  destined for clerics whose main activity had become the celebration of religious  offices, and which was carried out through luxuriant allegorical commentaries,  whose hermeneutic criteria today escape us, and which can sometimes take the  appearance of a veritable intellectual game. The example of Durandus of Mende’s  commentary on bells61 will suffice to show just how rarefied medieval liturgists  could become: The wood from which the bell is hung signifies the wood of the Lord’s cross, and  this is why bells are sometimes hung from the highest part of the tower, because  the cross had been announced by the earliest Fathers. The supports that attach  or nail together the parts of this wood are the oracles of the prophets. And the  iron joint that unites the bell to the wood means charity, by which the preacher  glories in being indissolubly nailed to the cross when he says, “Far be it from  me to think of glorying in anything else but the cross of Our Lord [Ga 6,14].”62 The writings of Pseudo-Dionysius probably had an important effect on the way this  era envisaged liturgical formation. In particular, the influence of the “hierarchical  doctor”, as he was sometimes known in the Middle Ages, may have facilitated a  vision in which the liturgy itself, and so liturgical formation, was seen as reserved  to clerics.

  

 



Moreover, the ninth century liturgist and bishop of Metz Amalarius, who died  around  $850^{64}.$ , is generally held to be at the origin of the numerous allegorical  commentaries of the liturgy that would carry on all the way to the nineteenth  century. Liturgical formation can be said at that point to have been transformed  into a rarefied understanding of the rites that re-interpreted the patristic tradition,  whilst drawing its symbolic understanding from Scripture, a symbolism that we  find baffling, since our own understanding of Scripture is moulded by modern  exegesis.

Although this allegorical method draws analogies from the Bible, more or less in  the way the poet Baudelaire sees correspondences between “scents, colours and  sounds”, Scripture is no longer its true referent. And so liturgical knowledge  becomes autonomous. As it moved from scriptural to ritual commentary in a  dynamic of increasing systemization, the allegorical approach came to be cut off  from the other interpretive forms used by the Fathers, and in the first place from  the literal sense, that is to say the rites’ raison d’être and their connection to the  liturgical action. Furthermore, because allegory was held to give the “spiritual”  or “mystical” meaning, foundations had been laid down that would later lead to  the separation between liturgy and spirituality. Allegory separated the liturgical  action from its interpretation, while spirituality came to find its sources outside  liturgical life.

All this could be taken as a pitiless critique of the medieval contribution to  liturgical formation; however, such one-sidedness calls for two remarks that not  only correct the imbalance, but also open two important new perspectives for  research on our theme.

The first comes from an unpublished doctoral dissertation in which Martin Morard  sheds light on the importance of the Psalter as a formative tool during the Middle  Ages. Through his evocation of a “quasi obsessive” interest in the Psalter, and  by the use he makes of the expression “psalmic culture” coined by Pierre Riché,  Morard seeks to identify how the psalms influenced practices, in particular those  that are related to formation: learning how to read and memorisation, as well as  the theological and liturgical formation of the clergy. The elements of this research  that interest us the most are found in its highlighting of the specific status of the  recitation of the Psalter, which according to Morard: … acted as a sacred technique at the service of psalmody executed with a  vicarious intention, at the service of the common good, as a realisation of  the Davidic prophecy that theologians considered to be “the” prophecy par  excellence, “almost as clear as the Gospel”.

 



However, Morard points out that this broke down in the twelfth century; with  the arrival of dialectical reasoning, textual analysis took precedence over such  practical ends, and became an element of scientific knowledge. This development  provoked a double rupture: among clergy, a dissociation between lectio and oratio,  and among laity, between the Psalter and devotional exercises.

This first corrective suggests to us liturgists the need to rethink the place of the  Psalter in contemporary liturgical formation. While exegetical studies of the psalms  abound, and although studies of their liturgical exegesis do exist74, it seems to me  that we lack studies showing how the psalms build up Christian liturgy, and how  psalmody can still form today all those who play their part in the Opus Dei.

The second corrective concerns the Dominican Jacques de Voragine, who died as  archbishop of Genoa in 1298, and whose great work known as the Legenda Aurea  is all too often reduced to a collection of legendary hagiographical tales, only good  for teaching us about the medieval imagination and for furnishing interpretative  keys for a number of iconographical programmes.

The reality could hardly be more different, for as Jacques Le Goff recently pointed  out in his preface to the text’s recent edition in the prestigious series “La Pléiade”76,  the Golden Legend is “one of the most important and most characteristic texts  of the Middle Ages”. The influence of this important medieval text was long  and considerable, and it brims with information that can be used in liturgical  formation. It could suffice to take the example of what the table of contents reveals  as to the importance of the notices about the different liturgical seasons in the  whole of the work. But the index shows that the very notices concerning the  saints contain much more about the liturgy, a technique that probably was all the  more effective as it articulated a liturgical content to the hagiographical narratives  that impressed themselves into people’s memories.



Thus, in his notice on St. Ambrose, Jacques de Voragine remarks: “He was so  strict in his ascetic practice that he fasted every day except Saturdays, Sundays,  and principal feast days.”79 And he attributes a major role in the establishment of  liturgical institutions to St. Jerome: In former times, people sang what they wanted in church; when the Emperor  Theodosius asked Pope Damasus to call upon a scholar to organise the  ecclesiastical offices… Jerome distributed the Psalter day by day, assigned its  own nocturn to each day, and he prescribed that at the end of each psalm the  “Glory be to the Father” should be said… Then he made a rational distribution  of the epistles and gospels over the course of the year, as well as whatever  concerns the chants of the office.



Obviously, this sort of information concerning the history of worship does not meet  our standards of historicity; but we can still hold that among all other medieval  works, liturgical summae included, the Golden Legend had a decisive influence  upon liturgical formation, simply because its way of handing on information in  narrative form unified “knowing” (savoir), “knowing how to do” (savoir-faire), and  above all, “knowing how to be” (savoir-être).

### 6.    FormaTIon For LITurgy under a modern sun

If I have emphasised the first three periods in this historical overview of liturgical  formation, it seems possible to limit its last stage to a few brief remarks. Of course  my main reason is to limit the length of my lecture; and many of the awaited  contributions to the Congress will certainly shed light on the riches of this period,  which bears the mark of a wide diversification. It was not possible for me to give its  rightful place to an ecumenical perspective that would have taken account of the  numerous developments of our different Christian communities.



But there also lies an affirmation and a diagnosis concerning our own day. Even  though we often speak of “postmodernity” or even of “ultramodernity”, it seems  to me that the question of liturgical formation still bears the mark of modernity.  Modernity’s starting point was the separation between Scripture, theology and  spirituality, a separation inherited from the Middle Ages and on which I have  founded the hypothesis developed in this lecture, and as it moved on from  this starting point, modernity has had its effects. We are still living within this  dynamic, even if more recent developments have modified the direction taken  since the Renaissance.

I have already underscored the importance of the Liturgical Movement as  the laboratory of the idea of liturgical formation, while recalling the need to  contextualise its work in the wider setting of modernity. But Early Modern  developments, that is evolutions that took place between the late fifteenth and the  seventeenth century brought about major shifts.

With the invention of printing, liturgical formation passed for the most part from  an oral to a written form, and took place more and more through an individual  study of books, whereas the medieval summae were still handed on collectively  through being taught by masters. It also came to be directed to an ever broader  public, including laity, and took the form of instruction situated outside the  liturgical action and giving great importance to historical elements. The title of  the main work of the Cluniac Benedictine Claude de Vert, a work that gave rise to  much controversy, can be taken as a sort of programme for the period: Explication  simple, littérale et historique des cérémonies de l’Église. It is no longer a matter of  interpreting liturgy by means of allegory in order to enter into a comprehension of  the action, but rather of “explaining” in order to know and understand.

At a time when Roman Catholics “attended” Mass, liturgical formation became an  “explanation of ceremonies”. Historical erudition, which sometimes rejected the  mystical interpretations of the Middle Ages, came to dominate the field. Little by  little, a desire of understanding rites and texts took root and transformed in turn  the way the act of formation was perceived. Henceforth, the royal road was one  of translation of texts and historical explanation of rites. From this point of view  there is a direct continuity between the first Liturgical Movement and modernity.

In another respect, the centralisation of liturgical authority in the Roman Catholic  world which was a consequence of the decisions of the Council of Trent and  the establishment of the Roman Congregation of Rites, would accentuate the  disciplinary aspect of the relationship with the liturgy; and so we speak of an era  of rubrics. Liturgical formation was deeply influenced by this narrowing of the  field of vision whose indelible mark is still perceptible today.

Nonetheless we should not forget other aspects which corrected the rigidity of  these approaches; the “explicative” paradigm and the insistence on discipline  that characterise the period are counterbalanced by an aesthetic search that  finds significant expressions in baroque architecture and in the period’s religious  music. Thus the effort expended on formation aims at the adherence not only of  reason but also of the heart, hence the centrality of art, liturgical singing, as well  as preaching.

At the same time, the implications of a spirituality that stressed a personal search  for interiority was detrimental to the communal experience of participation  in an action. This is why the twentieth century Liturgical Movement sought to  counterbalance these tendencies by re-enforcing the communal dynamic of the  celebration, and so placed formation for liturgy at the heart if its preoccupations.

But this model of formation aimed to undergird a somewhat militant project,  that of obtaining participation in the action that was “active”, “pious”, and even  “conscious”. So participation came to have a reciprocal relationship with formation;  it became the means and the setting of liturgical formation, while formation  became the prior condition for the progress and the authenticity of participation.  The model thus obtained is no doubt more an ideal than a reality, according to  which the liturgical celebration would become, or become anew if we take the  early Church into account, the essential setting for liturgical formation. It is a  model of formation for liturgy through its celebration.



### concLusIon

This panorama remains incomplete, and its multiple gaps still need to be filled in;  nonetheless, it should now be possible to come to a conclusion.

We began with a hypothesis: we can read historical developments in the light  of a diagnosis, that of a gradual separation beginning with the Middle Ages  between Scripture, theology and spirituality. By breaking leitourgia away from  the fundamental relationship it maintained in the Church’s first centuries with  martyria and diakonia, this separation often reduced Christian worship to a ritual  performance. In consequence, liturgical formation came to be conceived as the  transmission of knowledge on the subject liturgy.

Furthermore, even though at first this knowledge continued to be inseparable  from a spiritual wisdom concerning the action, thanks to the allegorical method  developed in the ninth century by Amalarius of Metz, and because this method  was founded on an interpretation of the rites which still drew its figures from  Scripture understood through analogy, later on, in particular after the birth of  early modernity, this knowledge became more and more technical, with the object  of correct execution of the rites by those whose task it was to perform them.

As a result of the Liturgical Movement, and above all thanks to a series of historical  studies, the liturgy gradually recovered its place not only as a science, but also as a  form of ecclesial life. The leaders of the Liturgical Movement have passed on to us  the legacy of the concern for formation which gave a decisive impetus to liturgical  studies in universities and other formation institutes in the Church.

However the enthusiasm aroused by the rediscovery of the documents and the  practices of the past led them to underestimate the phenomenon of cultural  distancing that was already taking place. So today, despite the admirable depth  and the richness of this heritage, it has become clear to many of us today that it  has become inaccessible, at least in part, to a large number of our contemporaries.

We all know how varied, indeed opposed, the reactions to this sort of diagnosis  can be. But we can at least draw two concluding thoughts from it.

First, the dream of a type of liturgical formation that would be able to find its  principal support within the celebration itself is today confronted with many  difficulties. The sort of historical panorama that we have just undertaken should  make us to realize that liturgical formation needs to find new routes for our post-  or ultra-modern era.



However, these new ways must seek to draw together Scripture, theology and  spirituality in such a way that formation not be subordinated to the sole end of  ritual performance, and above all that it be clearly understood to be summit and  source of Christian life.

Second, experts in pedagogy underline how much the digital world, that has only  really invaded our daily life since twenty years ago, has transformed in depth  not only the processes of learning and of gaining knowledge, but perhaps also  even the workings of our cognitive structures and our psyche. At the dawn of  modernity, the sixteenth-century reformers, Luther in particular, had an acute  perception of the importance of the liturgical experience for Christian liberty  (including an opposition to clerical pretensions). Today, new technologies have  created a relationship between the individual and knowledge according to the  mode of absolute autonomy: demand controls supply. In such a context, we  should ask ourselves if our historical overview does not invite us to give priority  to a conception of liturgical formation as a sort of ecclesial immersion bath, in  which individuals learn to appropriate the legacy of the past while creatively  reinventing it. At the least, this justifies that we at Societas Liturgica devote one of  our Congresses to the subject!

Translation:  Christopher Lazowski, OSB   Abbaye Saint-Wandrille
# Benchmarks for a History   of Liturgical Formation

## PATRICk PRéTOT, OSB\*

A s I begin, I would like to extend my warmest thanks to the council and those  who organized the Québec Congress for having invited me to speak today. I  was particularly touched by this invitation because, as you know, I had to stand  aside, much to my regret, from the organization of this Congress. So I would like  to express before you all my grateful affection towards Lizette Larson-Miller, for  this gesture of fellowship, and above all for having accepted to take over from me  as president. Our president elect Martin Stuflesser and all the members of the  council are of course included in this heartfelt thank-you. To be perfectly truthful,  I should mention that I hesitated before accepting this invitation to speak here in  Québec, because the underlying reasons for my well-founded decision not to come  to Würzburg remain. At the same time, I had to re-orient my future academic  obligations. And so it is with modesty and gratitude that I stand here before you  this morning. I am as aware of the extent of the honour that you bestow on me as  of my limits as to the task at hand.

### InTroducTIon

This presentation is obviously not going to attempt a synthesis of the history of  liturgical formation. The subject is not a totally unexplored area, but as far as my  research into the question has been able to enlighten me, it remains a subject that  has been not much studied for itself, even by liturgical historians. When the  question has indeed been taken up, it has been principally approached from the  point of view of contemporary problematics, and in the context of reflection upon  liturgical theology and pastoral practice.



We should also take note that in standard French studies of the history of education,  for example Henri-Irénée Marrou’s L’histoire de l’éducation dans l’Antiquité5 or  Pierre Riché’s (Marrou’s disciple and successor at the Sorbonne) classic works,  Éducation et culture dans l’Occident barbare and Ecoles et enseignement dans  $l e$   Haut Moyen  ${\hat{A}}g e^{6},$  the question of “liturgical formation” is not dealt with as such.  It does make an appearance in these authors’ studies of Christian Initiation7, but  the question of how laity and clergy were formed to liturgical life is not raised.  This relative silence is no doubt the result of separation between different fields of  academic study; although historians of liturgy and historians of education are not  unaware of each other’s existence, the dynamics of their respective studies have  never led them to construct a common object for their research.



Although the subject of liturgical history does not appear in the tables of contents  of historical publications, the question itself is far from absent. For example, Pierre  Riché, in a passage on “Elementary and Specialised Teaching”, points out that in  early medieval programmes of study, “reading and chant were complementary  subjects. Students memorized through song not only psalms, but also the short  Latin texts they were to study”. Then he adds this important remark: All children, even if they do not have good voices, were to learn to sing. When  Charlemagne made cantus part of his programme’s obligatory core content, his  object was both general formation and liturgical studies. The children were to  start participating in liturgical services, and to learn to sing on the basis of  psalms, hymns and canticles.



So it is already clear that, from the point of view of history, at least in the earlier  periods, liturgical formation is probably “hiding itself” in “general formation,” and  runs the risk of remaining invisible as much to historians as to liturgists. We may  even make the hypothesis that, over time, liturgical formation will become all the  more visible and necessary in so far as “general formation” loses its living link to  the liturgy. This first element can orient further research; at a time when processes  of transmission are breaking down, it would be helpful to dig deeper into this  fundamental aspect, in particular on the basis of documents like hagiographical  sources and canonical collections, with a view to coming to a better understanding  of the way the major benchmarks of liturgical experience were handed on in the  past, within Christian living. 



Having set this background, I shall now seek to shed light on the relationship played  out through history between, on one hand, liturgy in its celebration, and on the  other, what we can place under the approximate heading of “liturgical formation”. I  have just used the adjective “approximate” because it is clear to me that the notion  of “liturgical formation” hides a multitude of realities. It is to be hoped that the  different approaches that will become apparent during our Congress will help us  to delineate the subject with greater clarity, a subject whose foundational character  was underlined by the Liturgical Movement, and whose essential importance for  teachers of liturgy is now evident. The risk we run is that of a certain confusion if  we do not distinguish the varied realities lurking behind this notion.

At the least, a historical survey can help us to realize just how complex the idea  of “liturgical formation” is, by calling into question oversimplifications which  present it as a homogenous reality over the course of time. The very diversity  of the expressions used to designate it though history is a sign of a constant  plurality of practices as well as of understandings. Today, under the influence of  the Liturgical Movement, we usually speak of “liturgical formation” or “formation  for liturgy”, and while the term “mystagogy” has recently reappeared, in the Early  Modern period liturgists seem to have favoured the expressions “explanations”  and “instructions”.

In order to set the character of liturgical formation into its historical context, or  rather contexts, I shall divide my account into historical stages, with a view to  identifying the various actors, places and moments that play a role in the process, as  well as to working out as much as possible of its aims and methods. This approach  is not without risk. There will be no difficulty to find exceptions that do not fit into  the suggested outline. Moreover, in order fully to take into account the complexity  of the historical reality, it would be necessary to extend the investigation well  beyond what is possible here. However, in the absence of a more precise survey,  this approach may at least allow us to reach a general idea of the question.

To begin, a few methodological reflections seem necessary.

  Voir par exemple, Nicolas Le Tourneux, Instructions chrétiennes sur les sacrements et sur  les cérémonies avec lesquelles l’Eglise les administre, Paris, Hélie Josset, 1686 ; Jean-Jacques Ollier,  Explication des cérémonies de la grand’messe messe de paroisse selon l’usage romain, Paris, Langlois, 1687  ; et encore Louis-Albert Joly De Choin, Instructions sur le rituel, concernant la théorie et la pratique des  sacrements et de la morale, et tous les principes et décisions nécessaires aux curés, confesseurs, prédicateurs,  chanoines, bénéficiers, prêtres, ou simples clercs, Lyon, chez les frères Perisse Imprimeurs Libraires, 1778  (nombreuses rééditions).

### 1.    on The noTIon oF “LITurgIcaL FormaTIon”

We should begin by taking note that the notion of “liturgical formation”, meaning  the transmission of an intellectual knowledge concerning liturgy, is no doubt a  recent idea, born in the Liturgical Movement, where it became one of its most vital  aspects. It is not going too far to say that this concern for formation was at the  origin of the development between 1841 and 1962 of an educative project that today  seems almost inconceivable. Furthermore, this project reached the whole of the  Christian people, and was embodied in numerous publications, articles, sessions,  congresses, and above all in the publishing of missals like those of Dom Gaspar  Lefebvre (1880–1966), a Benedictine of the monastery of Saint-André-lez-Bruges13.

This provisional definition of “liturgical formation” as “the transmission of an  intellectual knowledge concerning liturgy” matches what comes to mind naturally  when we hear the expression. Of course liturgical formation goes well beyond this  approximate and provisional definition; hence two further important remarks are  called for.

In the first place, just as the pairs “worship” – “liturgy” in English, “Liturgie” –   in German, and “ $^{\alpha}K u l t^{\gamma}$ liturgie” – “culte” in French are not perfectly equivalent,  the term “formation” probably does not point to the same mental representations  in our different languages and cultures. This is partly because of differing traditions  as to the manner of teaching. But it is also rooted in the very nature of the matter  at hand. What is at issue here is not only a “knowledge about” (savoir sur) liturgy,  but also “knowing how to be” (savoir-être), a “knowing how to do” (savoir-faire),  and even an important element of “knowing how to live” (savoir-vivre). So we can  already see that the notion of formation masks a highly complex reality. It is thus  probable that the relative silence on this subject that I pointed out earlier results  from the difficulty it presents.



In the second place, this is true not only in space, but also over time. While it is  certainly true that the Liturgical Movement had a decisive impact on the idea of  formation, it is important to counterbalance this by pointing out that a concern  for liturgical formation is by no means limited to the contemporary period. What  is true of liturgical inculturation is also true of liturgical formation; the Church  has always been confronted with the necessity of transmitting its practices,  even though the ways of so doing have differed over the course of time. This  preoccupation existed long before the Liturgical Movement, but no doubt in a  different manner, and, above all, without the use of the word “formation”.

The importance of this preoccupation is striking; the learned publications on the  liturgy published in France throughout the seventeenth and eighteenth centuries  favour the word “instruction” and employ an explanatory paradigm. It is no secret  that Early Modernity bears the mark of the forces unleashed by Luther and the  other reformers, and of their desire that the disciples of Christ not be abandoned  to ignorance. This desire was echoed within the Roman Catholic Church in the  educational institutions developed after the Council of Trent; here in Québec we  can cite the work of Early Modern religious orders like the Society of Jesus and  the Ursulines. Liturgical historiography over the past century has perhaps overly  neglected this Early Modern period because of the justifiable attraction for the  primitive and patristic periods. But the Early Modern period has much to teach us  about the prehistory of that concern for formation that was so important for the  Liturgical Movement.

### 2.    a hIsTorIcaL seTTIng To PermIT a generaL overvIew 

As I said in my introduction, what follows is an overview divided into historical  periods with a view to identifying the principal stages of the history of liturgical  formation. But as Father Pierre-Marie Gy often pointed out, although division  into historical periods can be helpful, such divisions are always somewhat artificial  and so partly false. The divisions created never quite match the slow evolution  of practices and representations that characterize liturgical life. However, and  despite these necessary precautions, we shall adopt a division into four or five  main periods. The first period runs until the emergence of the earliest liturgical  institutions; hence it ends towards the middle of the fourth century. The second  period is the “high” patristic era. The third stage is the Middle Ages. It is important  here to bear in mind that the Middle Ages is anything but a homogenous block,  and with regard to formation, the fourteenth and fifteenth centuries are worlds  away from the ninth century; the emergence of universities in the thirteenth  century constitutes a sort of turning point.

Finally, the Early Modern / Modern period brings with it a new situation. The  Liturgical Movement was in part the product of this situation, and at the same  time the result of a sustained effort to move on, as can be seen in certain of Dom  Guéranger’s writings. So it would be possible to speak of a fifth period inaugurated  by the Liturgical Movement, because of the ways it transformed participation in  the liturgy as well as its methods of scientific investigation that would serve as a  basis for formation. Liturgical formation is durably marked by the development  of historical studies of liturgy following on from Msgr Duchesne. Indeed over  several decades liturgical formation was more or less identified with a knowledge  of liturgical history.



Within the setting of a paper delivered to this Congress, without going into detail,  I shall limit myself to calling attention to the essential structures of formation,  by identifying its principal actors, the context of the act of formation, and the  contents that were transmitted.

I am indebted for my hypothesis to an article on “practical and spiritual theology” by  René Marlé S.J., in which he suggests understanding the diversification of theology  over time as a process of progressive distinction and even separation between  Scripture, theology, liturgy and spirituality. In this evocative synthesis written  for a theology textbook, Marlé notes that in the patristic age, as well as during  part of the Middle Ages (the example of St. Anselm comes to mind), the form of  theological discourse was “above all, if not exclusively, a scriptural commentary.”  He adds that in this situation, “theology was directly spiritual, indeed it was also  pastoral, and developed spontaneously into prayer or exhortation.” But he goes on  to point out that since the later Middle Ages theology has taken: … the form of an often highly rarefied and subtle critique of the language of  faith and of the possibilities of discourse concerning God, and … in parallel,  a growing number of works and pamphlets, seeking, in varying registers, to  account for the experience of faith.



In other terms, since the Middle Ages, the splitting of theological reflection into  Scripture, theology and liturgy has isolated liturgy as a ritual practice generally  safeguarded by a body of rules destined to ensure its correct execution. This  splitting can be said to have “given sanctuary” to liturgy by cutting it off from  ethical responsibility and even, all too often, from the seeking of God. My  intention is now to consider our different historical periods from the point of view  of liturgical formation in the light of this hypothesis.



### 3.    The PrImITIve church

After the first generations follows the period of the apostolic Fathers, a time that  can be described as “pre-Constantinian”, covering the second and third centuries.  I shall group these two periods together, although an attentive reading of our past  president Gordon Lathrop’s recent book, The Four Gospels on Sunday, would invite  us to distinguish these two periods; the years during which the biblical canon  was slowly worked out obeyed a dynamic that escapes us for the most part, but  which also certainly left an essential mark on the formation of believers.

During these early periods, Christian institutions were still in gestation; their  embryonic forms were tributary to their regional contexts, and above all handed  on orally. Only a tiny quantity of evidence remains, which raises the permanent  risk of reading too much into it. We can of course cite important documents like  the Didachè, the letters of Ignatius of Antioch, Tertullian’s work on baptism,  or even, and especially, the Apostolic Tradition. Nonetheless, it is now widely  recognized how quickly any attempt to suggest general reconstructions runs up  against inevitable historiographical difficulties. However, it is probably true that  in this early period, the inheritance of Jewish institutions played a significant role,  so that the study of the Jewish origins of Christian worship may be illuminating.  But it is also clear that recent historiography on this subject lacks sufficient  consensus to draw firm conclusions.



During this first period, the first actors were without doubt the apostles, but  also, after them, and before the hierarchical structure: bishop/presbyterium and  deacons, emerged, didascales carried the new faith’s message. So it was the Gospel,  that is to say the proclamation of the Good News, in particular through the  Gospel stories, but also Paul’s letters, that constituted the first means of liturgical  formation. As Gordon Lathrop stressed in his opening address at the Würzburg  Congress, these were concerned not so much with saying something about  liturgy as with reforming assemblies so that they might live as authentic disciples  of Jesus. Of course, while the New Testament and the earliest Christian writings  were never textbooks for liturgical formation, we may still rapidly take note of two  examples: the first Letter of Paul to the Corinthians (1 Cor. 11:17-34) and Justin’s  First Apology.



When the apostle Paul wrote to believers in Corinth, we know that he undertook  a stern critique of their assemblies. He began by pointing out that these assemblies  “did more harm than good” (11:17), he did not hesitate to denounce the divisions  of the community (11:18-19), going as far as to discredit meetings that were not  what they were supposed to be; for he said that “when you all come together, it is  no longer the Lord’s supper that you take” (11:20).

At a later date, this passage would be too often interpreted in the light of verse 27:  “the one who will have eaten the bread or drunk the cup of the Lord unworthily  will have to answer for the body and blood of the Lord”, focalizing discernment  on the conscience of the individual who receives communion, an insistence which  has often kept believers away from participation in the eucharistic table. In  reality, Paul is recalling a received tradition (11: 23b-25) as an invitation to the  Corinthians not to pervert what is given in order to be lived in the Lord’s supper,  that is to say, “whenever you eat this bread and drink this cup, you proclaim the  death of the Lord until he comes”.

This is certainly an act of formation, one whose object is the authenticity of practices.  It is not something “about” the eucharist, but rather it seeks to foster Christian life;  and the eucharist is the criterion of the truth of Christian life. So we have here  a clear indication of how inseparable liturgical formation and ethical formation  are at this point in history. It is unsettling to realize that over history, respect for  traditions or for ritual regulations has at times coexisted with the sort of deviations  of meaning that Paul did not hesitate to denounce. For the Corinthians were  no doubt acting in accord with what they themselves thought they had received,  and indeed Paul does not ask them to modify the ritual order of their action. The  history of the Church can doubtless provide us with numerous examples of what  we could call perversions of liturgy. It bears repeating that the perversion is not in  the first place deviation from the ritual observance but a subversion of the action  inasmuch as it realises the saving work carried out by Christ’s Passover.

For Justin, liturgical formation seems to be closely linked to a vision of the Church  as a “priestly people”. This is why the essential setting of formation is the Sunday  assembly, not so much as the coming together of those who have followed the Way  as the manifestation of the priestly community of believers. For in the assembly  that meets on the Lord’s Day resound the “memoirs of the apostles” and “the  writings of the prophets”, and “the one who presides” “lifts up to heaven prayers  and thanksgivings”, and are shared not only “consecrated foods”, but other goods  in aid of “all who are in need”.

Hence it is clear that in this earliest period liturgical formation is undertaken  by those who announce the Gospel, and its content can be identified with that  Tradition which Paul says he has received from the Lord himself. This formation  takes place especially within the Sunday assembly and so is addressed to the whole  of the Christian people.

### 4.    The “hIgh” PaTrIsTIc PerIod

Of course, more than the preceding period, the age that has often been presented  as the golden age of the liturgy as well as that of the formation of the Christian  people for the liturgy, is the first that comes to mind when liturgical formation is  mentioned. However, as Robert Taft reminds us, liturgists must beware of making  the patristic era (fourth to sixth centuries) a “golden age”, even if it certainly is  a fascinating time for our subject. The fourth century, that saw the major  institutions of the Christian liturgy take form, in particular the catechumenate  and the development of the liturgical year, can be seen as expressing a primordial  concern for Christian formation; the catecheses that accompanied the process of  Christian initiation show that this concern for Christian formation is inseparable  from liturgical formation. Directly concerning our subject, the patristic  sacramental catecheses have even sometimes appeared to be a sort of normative  model for liturgical formation. The proximity of catechesis and liturgy during this  period can indeed lead us to wonder whether ancient catechesis was not primarily  liturgical formation, that is to say formation “by” and  $^{\mathrm{{c}}}{\mathrm{{in}}}^{\mathrm{{\nu}}},$  or “within”, liturgy.



The complexity of this period invites us to guard against an idealisation of the  fruits that may have been borne by the mystagogical teaching of the likes of Cyril  of Jerusalem, John Chrysostom or Theodore of Mopsuestia in the East, or the  baptismal catecheses of Ambrose of Milan and Augustine of Hippo. Nonetheless,  these outstanding texts remain sources of inspiration for the liturgical formation  of believers. We can take note of the fact that in France the classic distinction  between pre-baptismal and post-baptismal catechesis, along with the idea and  practice of mystagogy (at least in part)46 have been recovered as a result of a  general effort of catechetical renewal. But here again, the emphasis placed on  certain types of sources runs the risk of making us forget that liturgical formation  during this period also took many other routes, in particular the type of preaching  exemplified by Leo the Great or Gregory the Great49, what we call today liturgical  catechesis. But even though these documents must be accorded an utmost  importance, many others should be taken into account, and perhaps first among  them the hagiographical narratives that were beginning to come into circulation,  through which the example of the saints transmitted a “knowing how to be”  (savoir-être) of Christian practices, including ritual practices.



We need to listen to Augustine here, for there are good reasons for holding him as  the master of Christian pedagogy. For example, in his discussion of the formation  of catechumens, the author of De Catechizandis rudibus points out: Concerning the sacrament they are to receive, it is enough for the more gifted  to be told the meaning of the rites, but with the less intelligent it is necessary to  provide further explanations and comparisons, so that they do not treat what  they see lightly.

 



Furthermore, we should remember that in the well-known passage concerning  Christian signs in book III of De Doctrina christiana, Augustine underlines  Christian liberty in this matter; he emphasises that in the economy of the New  Covenant, the multitude of signs that were in effect at the time “of the patriarchs,  the prophets, and all the figures of the history of Israel” has been replaced by  $^\mathrm{{\sc\alpha}}\mathrm{{\boldmath~a~}~}$  small number” “that the Lord himself and the teaching of the apostles has handed  on to us” (he is making reference to “baptism and the sacrament of the Body and  Blood of the Lord”) and which he says are “very easy to accomplish, very august  in their meaning, and very holy in their observance”. Indeed, for Augustine, “all  Christians, when they receive them, know, after being instructed, what they refer  to, and are thereby led to revere them, not out of fleshly servitude, but, on the  contrary, with spiritual freedom”.

Augustinian liturgical formation thus appears in all its pertinence; at a time  when the tendency to confuse the mechanical application of rules with a true ars  celebrandi reappears here and there, the vision of the great bishop of Hippo is more  important than ever: Now, – says Augustine – as to follow the letter and to take the signs for the  realities they symbolise comes from weakness and bondage, so to misinterpret  the signs comes from being misled by error.



It is clear that the patristic period was an important time for liturgical formation.  In continuity with the preceding period, the liturgical assembly is still the veritable  site of formation; the importance placed on due order in assemblies and episcopal  complaints about the absenteeism that is beginning to make itself felt testify to  this. The subject matter is too vast to develop here; however, we can sum up what  can be learned from this early period in two points.

First, as with mystagogical catecheses, it was not so much a matter of liturgical  formation as we understand it today as formation for Christian living. This  formation made use of Scripture as its primary source in order to hand on the  fundamental elements of the faith rooted in liturgical experience.

Second, this “formation” was given in the setting of the Christian assembly, normally  by bishops, or by priests or deacons in their name; there was thus an essential link  between liturgical formation and the service of the community. Liturgical formation  was not anyone’s specialty, but was part of the duties of all pastors.

### 5.    medIevaL LIghT and shadows

I had originally planned to spend more time on this particular period, because as  I prepared this lecture, my preliminary research led me to enriching discoveries  concerning the Middle Ages. As well, among historians, medievalists have paid  more attention than most to the question of the transmission of knowledge. But  also because it is particularly during this period that the concern with liturgical  formation begins to appear and find expression everywhere.

In conformity with this lecture’s general hypothesis, that of a splitting up of  Scripture, theology and spirituality beginning in the Middle Ages, the period  can be described as that of the veritable birth of formation “for” liturgy. Whereas  during the patristic period theology, and hence liturgical formation, took place  principally in the setting of the Christian assembly, and was thus an essentially  pastoral act, the gradual appearance of a specialised body of theologians would  little by little lead theological reflection to leave its initial community setting, and  to become the prerogative first of the masters of the monastic cloisters, then of the  masters of the university quadrangles.

Medieval writings on liturgy constitute an immensely rich body of works that took  form progressively from the De officiis of Isidore of Seville at the beginning of  the seventh century until the Rationale of Durandus of Mende (towards 1230-

1296) at the end of the thirteenth. We cannot do more here than evoke its scale,  without even being able to cite its principal elements. These liturgical summae,  a term we can use without fear of exaggeration, so clear is the intention of an  encyclopaedic presentation, are the counterparts of the theological summae of the  scholastic age. Formation for liturgy here takes the form of the transmission of  a rarefied knowledge of the rites and their meaning, a transmission which was  destined for clerics whose main activity had become the celebration of religious  offices, and which was carried out through luxuriant allegorical commentaries,  whose hermeneutic criteria today escape us, and which can sometimes take the  appearance of a veritable intellectual game. The example of Durandus of Mende’s  commentary on bells61 will suffice to show just how rarefied medieval liturgists  could become: The wood from which the bell is hung signifies the wood of the Lord’s cross, and  this is why bells are sometimes hung from the highest part of the tower, because  the cross had been announced by the earliest Fathers. The supports that attach  or nail together the parts of this wood are the oracles of the prophets. And the  iron joint that unites the bell to the wood means charity, by which the preacher  glories in being indissolubly nailed to the cross when he says, “Far be it from  me to think of glorying in anything else but the cross of Our Lord [Ga 6,14].”62 The writings of Pseudo-Dionysius probably had an important effect on the way this  era envisaged liturgical formation. In particular, the influence of the “hierarchical  doctor”, as he was sometimes known in the Middle Ages, may have facilitated a  vision in which the liturgy itself, and so liturgical formation, was seen as reserved  to clerics.

  

 



Moreover, the ninth century liturgist and bishop of Metz Amalarius, who died  around  $850^{64}.$ , is generally held to be at the origin of the numerous allegorical  commentaries of the liturgy that would carry on all the way to the nineteenth  century. Liturgical formation can be said at that point to have been transformed  into a rarefied understanding of the rites that re-interpreted the patristic tradition,  whilst drawing its symbolic understanding from Scripture, a symbolism that we  find baffling, since our own understanding of Scripture is moulded by modern  exegesis.

Although this allegorical method draws analogies from the Bible, more or less in  the way the poet Baudelaire sees correspondences between “scents, colours and  sounds”, Scripture is no longer its true referent. And so liturgical knowledge  becomes autonomous. As it moved from scriptural to ritual commentary in a  dynamic of increasing systemization, the allegorical approach came to be cut off  from the other interpretive forms used by the Fathers, and in the first place from  the literal sense, that is to say the rites’ raison d’être and their connection to the  liturgical action. Furthermore, because allegory was held to give the “spiritual”  or “mystical” meaning, foundations had been laid down that would later lead to  the separation between liturgy and spirituality. Allegory separated the liturgical  action from its interpretation, while spirituality came to find its sources outside  liturgical life.

All this could be taken as a pitiless critique of the medieval contribution to  liturgical formation; however, such one-sidedness calls for two remarks that not  only correct the imbalance, but also open two important new perspectives for  research on our theme.

The first comes from an unpublished doctoral dissertation in which Martin Morard  sheds light on the importance of the Psalter as a formative tool during the Middle  Ages. Through his evocation of a “quasi obsessive” interest in the Psalter, and  by the use he makes of the expression “psalmic culture” coined by Pierre Riché,  Morard seeks to identify how the psalms influenced practices, in particular those  that are related to formation: learning how to read and memorisation, as well as  the theological and liturgical formation of the clergy. The elements of this research  that interest us the most are found in its highlighting of the specific status of the  recitation of the Psalter, which according to Morard: … acted as a sacred technique at the service of psalmody executed with a  vicarious intention, at the service of the common good, as a realisation of  the Davidic prophecy that theologians considered to be “the” prophecy par  excellence, “almost as clear as the Gospel”.

 



However, Morard points out that this broke down in the twelfth century; with  the arrival of dialectical reasoning, textual analysis took precedence over such  practical ends, and became an element of scientific knowledge. This development  provoked a double rupture: among clergy, a dissociation between lectio and oratio,  and among laity, between the Psalter and devotional exercises.

This first corrective suggests to us liturgists the need to rethink the place of the  Psalter in contemporary liturgical formation. While exegetical studies of the psalms  abound, and although studies of their liturgical exegesis do exist74, it seems to me  that we lack studies showing how the psalms build up Christian liturgy, and how  psalmody can still form today all those who play their part in the Opus Dei.

The second corrective concerns the Dominican Jacques de Voragine, who died as  archbishop of Genoa in 1298, and whose great work known as the Legenda Aurea  is all too often reduced to a collection of legendary hagiographical tales, only good  for teaching us about the medieval imagination and for furnishing interpretative  keys for a number of iconographical programmes.

The reality could hardly be more different, for as Jacques Le Goff recently pointed  out in his preface to the text’s recent edition in the prestigious series “La Pléiade”76,  the Golden Legend is “one of the most important and most characteristic texts  of the Middle Ages”. The influence of this important medieval text was long  and considerable, and it brims with information that can be used in liturgical  formation. It could suffice to take the example of what the table of contents reveals  as to the importance of the notices about the different liturgical seasons in the  whole of the work. But the index shows that the very notices concerning the  saints contain much more about the liturgy, a technique that probably was all the  more effective as it articulated a liturgical content to the hagiographical narratives  that impressed themselves into people’s memories.



Thus, in his notice on St. Ambrose, Jacques de Voragine remarks: “He was so  strict in his ascetic practice that he fasted every day except Saturdays, Sundays,  and principal feast days.”79 And he attributes a major role in the establishment of  liturgical institutions to St. Jerome: In former times, people sang what they wanted in church; when the Emperor  Theodosius asked Pope Damasus to call upon a scholar to organise the  ecclesiastical offices… Jerome distributed the Psalter day by day, assigned its  own nocturn to each day, and he prescribed that at the end of each psalm the  “Glory be to the Father” should be said… Then he made a rational distribution  of the epistles and gospels over the course of the year, as well as whatever  concerns the chants of the office.



Obviously, this sort of information concerning the history of worship does not meet  our standards of historicity; but we can still hold that among all other medieval  works, liturgical summae included, the Golden Legend had a decisive influence  upon liturgical formation, simply because its way of handing on information in  narrative form unified “knowing” (savoir), “knowing how to do” (savoir-faire), and  above all, “knowing how to be” (savoir-être).

### 6.    FormaTIon For LITurgy under a modern sun

If I have emphasised the first three periods in this historical overview of liturgical  formation, it seems possible to limit its last stage to a few brief remarks. Of course  my main reason is to limit the length of my lecture; and many of the awaited  contributions to the Congress will certainly shed light on the riches of this period,  which bears the mark of a wide diversification. It was not possible for me to give its  rightful place to an ecumenical perspective that would have taken account of the  numerous developments of our different Christian communities.



But there also lies an affirmation and a diagnosis concerning our own day. Even  though we often speak of “postmodernity” or even of “ultramodernity”, it seems  to me that the question of liturgical formation still bears the mark of modernity.  Modernity’s starting point was the separation between Scripture, theology and  spirituality, a separation inherited from the Middle Ages and on which I have  founded the hypothesis developed in this lecture, and as it moved on from  this starting point, modernity has had its effects. We are still living within this  dynamic, even if more recent developments have modified the direction taken  since the Renaissance.

I have already underscored the importance of the Liturgical Movement as  the laboratory of the idea of liturgical formation, while recalling the need to  contextualise its work in the wider setting of modernity. But Early Modern  developments, that is evolutions that took place between the late fifteenth and the  seventeenth century brought about major shifts.

With the invention of printing, liturgical formation passed for the most part from  an oral to a written form, and took place more and more through an individual  study of books, whereas the medieval summae were still handed on collectively  through being taught by masters. It also came to be directed to an ever broader  public, including laity, and took the form of instruction situated outside the  liturgical action and giving great importance to historical elements. The title of  the main work of the Cluniac Benedictine Claude de Vert, a work that gave rise to  much controversy, can be taken as a sort of programme for the period: Explication  simple, littérale et historique des cérémonies de l’Église. It is no longer a matter of  interpreting liturgy by means of allegory in order to enter into a comprehension of  the action, but rather of “explaining” in order to know and understand.

At a time when Roman Catholics “attended” Mass, liturgical formation became an  “explanation of ceremonies”. Historical erudition, which sometimes rejected the  mystical interpretations of the Middle Ages, came to dominate the field. Little by  little, a desire of understanding rites and texts took root and transformed in turn  the way the act of formation was perceived. Henceforth, the royal road was one  of translation of texts and historical explanation of rites. From this point of view  there is a direct continuity between the first Liturgical Movement and modernity.

In another respect, the centralisation of liturgical authority in the Roman Catholic  world which was a consequence of the decisions of the Council of Trent and  the establishment of the Roman Congregation of Rites, would accentuate the  disciplinary aspect of the relationship with the liturgy; and so we speak of an era  of rubrics. Liturgical formation was deeply influenced by this narrowing of the  field of vision whose indelible mark is still perceptible today.

Nonetheless we should not forget other aspects which corrected the rigidity of  these approaches; the “explicative” paradigm and the insistence on discipline  that characterise the period are counterbalanced by an aesthetic search that  finds significant expressions in baroque architecture and in the period’s religious  music. Thus the effort expended on formation aims at the adherence not only of  reason but also of the heart, hence the centrality of art, liturgical singing, as well  as preaching.

At the same time, the implications of a spirituality that stressed a personal search  for interiority was detrimental to the communal experience of participation  in an action. This is why the twentieth century Liturgical Movement sought to  counterbalance these tendencies by re-enforcing the communal dynamic of the  celebration, and so placed formation for liturgy at the heart if its preoccupations.

But this model of formation aimed to undergird a somewhat militant project,  that of obtaining participation in the action that was “active”, “pious”, and even  “conscious”. So participation came to have a reciprocal relationship with formation;  it became the means and the setting of liturgical formation, while formation  became the prior condition for the progress and the authenticity of participation.  The model thus obtained is no doubt more an ideal than a reality, according to  which the liturgical celebration would become, or become anew if we take the  early Church into account, the essential setting for liturgical formation. It is a  model of formation for liturgy through its celebration.



### concLusIon

This panorama remains incomplete, and its multiple gaps still need to be filled in;  nonetheless, it should now be possible to come to a conclusion.

We began with a hypothesis: we can read historical developments in the light  of a diagnosis, that of a gradual separation beginning with the Middle Ages  between Scripture, theology and spirituality. By breaking leitourgia away from  the fundamental relationship it maintained in the Church’s first centuries with  martyria and diakonia, this separation often reduced Christian worship to a ritual  performance. In consequence, liturgical formation came to be conceived as the  transmission of knowledge on the subject liturgy.

Furthermore, even though at first this knowledge continued to be inseparable  from a spiritual wisdom concerning the action, thanks to the allegorical method  developed in the ninth century by Amalarius of Metz, and because this method  was founded on an interpretation of the rites which still drew its figures from  Scripture understood through analogy, later on, in particular after the birth of  early modernity, this knowledge became more and more technical, with the object  of correct execution of the rites by those whose task it was to perform them.

As a result of the Liturgical Movement, and above all thanks to a series of historical  studies, the liturgy gradually recovered its place not only as a science, but also as a  form of ecclesial life. The leaders of the Liturgical Movement have passed on to us  the legacy of the concern for formation which gave a decisive impetus to liturgical  studies in universities and other formation institutes in the Church.

However the enthusiasm aroused by the rediscovery of the documents and the  practices of the past led them to underestimate the phenomenon of cultural  distancing that was already taking place. So today, despite the admirable depth  and the richness of this heritage, it has become clear to many of us today that it  has become inaccessible, at least in part, to a large number of our contemporaries.

We all know how varied, indeed opposed, the reactions to this sort of diagnosis  can be. But we can at least draw two concluding thoughts from it.

First, the dream of a type of liturgical formation that would be able to find its  principal support within the celebration itself is today confronted with many  difficulties. The sort of historical panorama that we have just undertaken should  make us to realize that liturgical formation needs to find new routes for our post-  or ultra-modern era.



However, these new ways must seek to draw together Scripture, theology and  spirituality in such a way that formation not be subordinated to the sole end of  ritual performance, and above all that it be clearly understood to be summit and  source of Christian life.

Second, experts in pedagogy underline how much the digital world, that has only  really invaded our daily life since twenty years ago, has transformed in depth  not only the processes of learning and of gaining knowledge, but perhaps also  even the workings of our cognitive structures and our psyche. At the dawn of  modernity, the sixteenth-century reformers, Luther in particular, had an acute  perception of the importance of the liturgical experience for Christian liberty  (including an opposition to clerical pretensions). Today, new technologies have  created a relationship between the individual and knowledge according to the  mode of absolute autonomy: demand controls supply. In such a context, we  should ask ourselves if our historical overview does not invite us to give priority  to a conception of liturgical formation as a sort of ecclesial immersion bath, in  which individuals learn to appropriate the legacy of the past while creatively  reinventing it. At the least, this justifies that we at Societas Liturgica devote one of  our Congresses to the subject!

Translation:  Christopher Lazowski, OSB   Abbaye Saint-Wandrille
# Crossability of Brassica carinata and B. tournefortii, and  Cytomorphology of Their  $\mathsf{F}_{\gamma}$  Hybrid

B. R. Choudhary and P. Joshi

Agricultural Research Station (SK RAU), Mandor-Jodhpur 342 304, India  Received June 1, 2012; accepted October 14, 2012 Summary Reciprocal crosses between Brassica carinata (BBCC, 2n=34) and B. tournefortii (TT,  ) were attempted in order to determine crossability between the species, carry out chromo- $2n{=}20^{\circ}$  somal associations among the B, C and T genomes, and to study morphology of the interspeciﬁc hy- brids. A hybrid between these 2 species was obtained only when  $B$ . carinata was used as a female  parent. Morphologically, the hybrid was intermediate between the progenitor species but its leaves  resembled  $B$ . carinata. The  $\mathrm{F}_{1}$  plant was tall but grew very slowly. The hybrid was almost male   sterile, showing only  $2.3\%$  pollen stainability. Meiotic studies of trigenomic triploid hybrid (BCT,  ) showed various chromosome conﬁgurations including quadrivalents (0–1), trivalents (0–2),  $2n{=}27.$  bivalents (2–9) and univalents (5–23). Only 5 univalents were recorded to have occurred; otherwise  trivalents and bivalents were observed in pollen mother cells of the BCT hybrid. A few chromo- somal association could be expected through allosyndesis which suggests the possibility of transfer- ring genes across the species through interspeciﬁc hybridization. 

 



Key words  Brassica carinata, Brassica tournefortii, Crossability, Cytomorphology, Interspeciﬁc  hybrid.

Ethiopian or Abyssinian mustard (Brassica carinata A. Braun,  $2n{=}34$ ) possesses many desir- able characteristics that are rare in the other Brassica oil crops: heat and drought tolerance, toler- ance to various biotic and abiotic stresses and availability of yellow-seeded germplasm (Jiang et al.  2007). Inspite of these positive attributes, the crop suffers from many agronomic limitations like  longer crop duration, poor harvest index, low oil content etc., whereas  $B$ . tournefortii  $(2n{=}20)$ ,  known variously as wild turnip or Sahara mustard or Asian mustard or Tournefortʼs mustard or  Tournefortʼs Birdrape, grown sporadically in a few pockets of arid and semi-arid areas, has a strat- egy of early and quick growth because this effectively captures available soil moisture, enables it to  build a canopy, reproduce and mature before neighbouring species (Minnich and Sanders 2000). It  has also been reported as having a good source of tolerance against aphids (Singh et al. 1965) and  drought (West and Nabhan 2002). However, lower test weight ( $\mathrm{2-3\,g}/1000$  seed) and susceptibility  to downy mildew (Rajpurohit and Choudhary 1995) are the major constraints in the spread of this  species. 

Interspeciﬁc hybridization has great potential for the improvement of Brassica. Besides the  transfer of useful traits across the species in genus Brassica (Roy 1984, Rahman 2001, Rygulla et  al. 2007), such crosses were widely used to create genetic variability (Prakash 1973a), genetic   diversity (Choudhary and Joshi 2001) and to synthesize artiﬁcially new species such as  Brassicoraphanus (Tokumasu 1970) Brassica napocampestris (McNaughton 1973) and  Raphanofortii (Choudhary et al. 2000a). Both  $B$ . carinata and  $B$ . tournefortii species have their  own advantage and deﬁciencies. Therefore, the present study was undertaken to examine crossabil- ity between  $B$ . carinata and  $B_{i}$ . tournefortii and to observe the meiotic behaviour of the interspeciﬁc   hybrid of these species as it suggests the possibility of genetic introgressions for crop improve- $\mathrm{F}_{1}$  ments which take place through chromosomal interchange. 



Materials and methods Two genotypes of  $B$ . tournefortii (TT,  $2n{=}20$ ) — RBT 58 and RBT 61 — and 1 of  $B_{i}$ . carinata  (BBCC,  $2n{=}34.$ ) — BCN selected from the germplasm collection of the Agricultural Research  Station, Mandor-Jodhpur, India were used in the hybridization programme. 



The reciprocal interspeciﬁc crosses were attempted involving the selected genotypes of both  the species under ﬁeld conditions. Emasculation and pollination were conducted as per the given  procedure. Immature ﬂower buds of appropriate size were emasculated by removal of anthers in the  evening and bagged with butter paper bags. In the following morning, stigma of these buds were  pollinated with pollen of the desired parent and the racemes were re-enclosed with bags to prevent  cross pollination. Seeds obtained from crosses and parents were grown in the ﬁeld for evaluation.  Morphological comparisons were made, and the meiosis of the interspeciﬁc hybrid was studied. 

For meiotic analysis, young ﬂower buds were ﬁxed in freshly prepared solution of propionic  acid: absolute ethanol  $(1:3)$  containing ferric chloride as a mordent. After  $24\,\mathrm{h}$  the rinsed ﬂower  buds were stored in  $70\%$  ethanol until slides were made by the acetocarmine  $(1\%)$  squash tech- nique. Chromosome associations were observed at diakinesis/metaphase I. Pollen fertility of the   hybrid and parents was estimated by pollen grain stainability in acetocarmine  $(1\%)$ .

Results Crossability The seed setting was almost double in the cross  $B$ . carinata $\times B$ . tournefortii compared to its re- ciprocal combination. Out of 112 ﬂowers buds of  $B$ . carinata pollinated with pollen of  $B_{i}$ . tourne- fortii, 22 seeds were obtained giving rise to only a single hybrid plant. On contrary, in the recipro- cal combination, out of 122 buds of  $B$ . tournefortii pollinated by  $B_{i}$ . carinata pollen, only 11 seeds  were obtained which did not yield any true hybrids. 



Morphological features of  $F_{l}$  hybrids The phenotype of the hybrid was intermediate between parent species. The size and shape of  leaves of the hybrid were closer to  $B_{i}$ . carinata. They were obovate-lanceolate, petiolate, pinnatiﬁed  to pinnatipartite with entire to slightly dentate margin, obtuse tip, sparsely hairy and dark green in  colour. The ﬂowers were medium in size with pale yellow petals but their shape was similar to  $B$ .  tournefortii. The hybrid was tall but grew very slowly and took as many as  $95\,\mathrm{d}$  to ﬂower and  $168\,\mathrm{d}$   to mature completely (Table 1). The  $\mathrm{F_{1}}$  plant exhibited enhancement in values of attributes like pri- mary and secondary branches, main raceme length, siliquae on main raceme and siliquae per plant.  On the other hand, siliqua length, seeds per siliqua, seed weight and seed yield were recorded much  lower in the hybrid compared to either of the parents. Few seeds were set in the  $\mathrm{F}_{1}$  plant due to  open pollination. However, no seed set was found under self-pollination or in backcrosses. 

Meiotic characteristics of $\boldsymbol{{F}}_{l}$  hybrids The chromosome associations observed in pollen mother cells (PMCs) of the trihaploid BCT  hybrid  $(2n{=}27)$  are given in Table 2 and some representative cells are shown in Figs. 1–3. A total of  59 PMCs of the hybrid were analyzed at diakinesis/metaphase I. At least 2 bivalents were observed  in all the PMCs, but the maximum of 9 bivalents were observed in  $17\%$  of cells (Fig. 2). Many of  them exhibited chiasmatic conﬁgurations (Fig. 2). Multivalents in the form of trivalents (0–2) and  Discussion The hybrid from cross .  $B$ carinata ${\times}B$ . tournefortii was obtained only when  $B$ . carinata was  used as a female parent. This was not anticipated as in the majority of interspeciﬁc crosses reported  in Brassica, there was greater success when the species with the higher basic chromosome number  was used as a female (Quazi 1988, Choudhary and Joshi 1999). Nishiyama et al. (1991) reported  that crosses between tetraploid and diploid species were partly successful, while the reciprocal, i.e.  diploid⊗tetraploid, usually failed. However, the occurrence of the hybrid in the direction of .  $B$  carinata⊗B. tournefortii in the present study is in contradiction to Harberdʼs theory that interspe- ciﬁc crosses with  $B$ . tournefortii were only possible when it was used as a female parent (Harberd  1976).

 



The sexual hybrid between  $B$ . carinata and  $B$ . tournefortii was reported for the ﬁrst time by us  (Joshi and Choudhary 1999). The hybrid plant was observed to be an intermediate in terms of many  phenotypic features of both the parents. This is an advantage, since it would allow better selection  for speciﬁc attributes in segregating progenies.

Meiotic studies of trigenomic triploid hybrid BCT  $(2n{=}27)$ ) revealed various chromosome con- ﬁgurations including quadrivalents (0–1), trivalents (0–2) and bivalents (2–9). Based on earlier  studies on haploid genomes of B (Prakash 1973b), C (Armstrong and Keller 1982) and T (Prakash  1974), a total of 7 bivalents were theoretically possible due to autosyndesis in these 3 genomes.  Hence, out of the maximum of 9 bivalents observed in the present hybrid, 2 could be expected to be  due to allosyndesis among the B, C and T genomes. Mizushima (1968) noticed a maximum of 3 bi- valents in a TC hybrid and opined that one of them might be due to allosyndesis between these 2  genomes, while Ljungberg et al. (1993) observed a maximum of  $1\ \mathrm{III}{+}3\ \mathrm{II}{+}10\ \mathrm{I}$  in a similar hybrid  and suggested the possibility of allosyndetic pairing in T and C genomes. Narain and Prakash  (1972) reported a maximum of 3 bivalents in a BT hybrid but did not say anything regarding the  nature of origin of such bivalents. Sarla and Raut (1991) found a maximum of 1 III+5 II+4 I in BC  hybrid and demonstrated homology between the B and C genomes. Pairing in the present triploid  hybrid has been assumed to occur mostly between the B and C genomes. However, less than 10  univalents (possibly of T genome) were counted in  $45.8\%$  of all PMCs, suggesting that some T  chromosomes of .  $B$ tournefortii might have been involved in the pairing with B and C chromo- somes.



The high level of pairing observed in the hybrid might be due to auto/allosyndesis pairing  within and among the genomes. This might be attributed to structural similarities and duplications  of some of the chromosomes (Röbbelen 1960). However, the heteromorphic appearance of biva- lents in some of the PMCs observed and the presence of only 5 univalents, instead of the  $27\ \mathrm{ex}.$ - pected, indicated the extent of homology among B, C and T genomes. Bridges at anaphase I ob- served in the present material might have resulted from pairing between partially chromosomes of  the parent genomes. Such inter-genomic pairing in Brassica hybrid was also observed by Howard  (1938). The occurrence of multivalent association at a high frequency in the hybrid studied indi- cated that parts of the chromosomes of these genomes possess sufﬁcient afﬁnity to allow multiva- lent association (Newell et al. 1984). This showed that some of chromosomes of these 3 genomes  had duplicate archetypes and conﬁrmed the hypothesis of the secondary polyploidy origin of the B,  C and T genomes. Secondly, the lower average chromosome association observed in the present  BCT hybrid, compared to that reported in an ABC hybrid by Choudhary et al. (2000b), suggested  that the T genome was more distantly related than A to both the B and C genomes. Inter-genomic  chromosome associations noticed in the hybrid indicated the possibility of transferring genes across  the species through interspeciﬁc hybridization. 

Acknowledgements The authors would like to thank the late Dr. S. Prakash, New Delhi, for his help during this in- vestigation. 



References

Armstrong, K. C. and Keller, W. A. 1982. Chromosome pairing in haploids of Brassica oleracea. Can. J. Genet. Cytol. 24:  735–739. Choudhary, B. R. and Joshi, P. 1999. Interspeciﬁc hybridization in Brassica. In: Proceedings of the  $10^{\mathrm{th}}$  International  Rapeseed Congress. Canberra, CD-ROM.  — and — 2001. Genetic diversity in advanced derivatives of Brassica interspeciﬁc hybrids. Euphytica 121: 1–7. 

—, — and Singh, K. 2000a. Synthesis, morphology and cytogenetics of Raphanofortii (TTRR, 2n=38): a new amphidiploid  of hybrid Brassica tournefortii (TT,  $2n{=}20$ )⊗Raphanus caudatus (RR,  $2n{=}18$ ). Theor. Appl. Genet. 101: 990–999. —, — and Ramarao, S. 2000b. Interspeciﬁc hybridization between Brassica carinata and B. rapa. Plant Breed. 119: 417– 420. Harberd, D. J. 1976. Cytotaxonomic studies of Brassica and related genera. In: Vaughan, J. G., MacLeod, A. J. and Jones,  B. M. G. (eds.). The Biology and Chemistry of the Cruciferae. Academic Press, London. pp. 47–68. Howard, H. W. 1938. The fertility of amphidiploids from the cross Raphanus sativum⊗Brassica oleracea. J. Genet. 36:  239–273. Jiang, Y., Tian, E., Li, R., Chen, L. and Meng, J. 2007. Genetic diversity of Brassica carinata with emphasis on the inter- speciﬁc crossability with B. rapa. Plant Breed. 126: 487–491. Joshi, P. and Choudhary, B. R., 1999. Interspeciﬁc hybridization in Brassica. I. Brassica carinata⊗B. tournefortii. In:  Proceedings of  $10^{\mathrm{th}}$  International Rapeseed Congress. Canberra, CD-ROM.  Ljungberg, A., Cheng, B. F. and Heneen, W. K. 1993. Investigation of hybrids between Brassica tournefortii Govan and   B. alboglabra Baily. Sveriges Utsadesforenings Tidskrift 103: 191–197. McNaughton, I. H. 1973. B. rassica napocampestris L ( $2n{=}58.$ ) I. Synthesis, cytology, fertility and general considerations.  

Euphytica 22: 301–309.

Minnich, R. A. and Sanders, A. C. 2000. Brassica tournefortii Gouan. In: Bossard, C. C., Randall, J. M. and Hoshovsky, M.  C. (eds.). Invasive plants of Californiaʼs wildlands. University of California Press, Berkeley and Los Angeles,  California.

Mizushima, U. 1968. Phylogenetic studies on some wild Brassica species. Tohoku Journal of Agricultural Research 19:  83–99.

Narain, A. and Prakash, S. 1972. Investigations on the artiﬁcial synthesis of amphidiploids of Brassica tournefortii Gouan  with the other elementary species of Brassica. 1. Genomic relationships. Genetica 43: 90–97.

Newell, C. A., Rhoads, M. L. and Bidney, D. L. 1984. Cytogenetic analysis of plants regenerated from tissue explants and  mesophyll protoplasts of winter rape, Brassica napus L. Can. J. Genet. Cytol. 26: 752–761.

Nishiyama, I., Sarashima, M. and Matsuzawa, Y. 1991. Critical discussion on abortive interspeciﬁc crosses in Brassica.  Plant Breed. 107: 288–302.

Prakash, S. 1973a. Non-homologous meiotic pairing in the A and B genomes of in Brassica: its breeding signiﬁcance in the  production of variable amphidiploids. Genet. Res. 21: 133–137. 

— 1973b. Haploidy in Brassica nigra Koch. Euphytica 22: 613–614. — 1974. Haploid meiosis and origin of Brassica tournefortii Gouan. Euphytica 23: 591–595. Quazi, M. H. 1988. Interspeciﬁc hybrids between Brassica napus L. and B. oleracea L. developed by embryo cul- ture. Theor. Appl. Genet. 75: 309–318.

Rahman, M. H. 2001. Production of yellow-seeded Brassica napus through interspeciﬁc crosses. Plant Breed. 120: 463– 472 Rajpurohit, T. S. and Choudhary, B. R. 1995. Downy mildew (Peronospora parasitica) of wild turnip (Brassica tournefor- tii), a new record from Rajasthan. Indian Journal of Agricultural Sciences 65: 377.

Roy, N. N. 1984. Interspeciﬁc transfer of Brassica juncea type high blackleg resistance to B. napus. Euphytica 33: 295– 303.

Röbbelen, G. 1960. Beitrage zur Analyse des Brassica Genomes. Chromosoma 11: 205–228. Rygulla, W., Snowdon, R. J., Eynck, C., Koopmann, B., von Tiedemann, A., Lühs, W. and Friedt, W. 2007. Broadening the  genetic basis of Verticillium longisporum resistance in Brassica napus by interspeciﬁc hybridization.   Phytopathology 97:1391–1396.

Sarla, N. and Raut, R. N. 1991. Cytogenetical studies on Brassica nigra $\times B$ . oleracea hybrids. Indian J Genet Plant Breed  51: 408–413.

Singh, S. R., Narain, A., Srivastava, K. P. and Siddiqui, J. A. 1965. Fecundity of the mustard aphid (Lipaphis erismi Kalt)  on different rapes and mustard species. J Oilseeds Res 9: 215–219.

Tokumasu, S. 1970. Intergeneric hybrids between Brassica japonica and Raphanus sativus. Memoirs of the College of  Agriculture Ehime University 14: 285–302.

West, P. and Nabhan, G. P. 2002. Invasive plants: their occurrence and possible impact on the central Gulf Coast of Sonara  and the Midriff Islands in the sea of Cortes. In: Tellman, B. (ed.). Invasive exotic species in the Sonoran Desert re- gion. The University of Arizona Press and the Arizona-Sonora Desert Museum, Tucson, Arizona.
# Crossability of Brassica carinata and B. tournefortii, and  Cytomorphology of Their  $\mathsf{F}_{\gamma}$  Hybrid

B. R. Choudhary and P. Joshi

Agricultural Research Station (SK RAU), Mandor-Jodhpur 342 304, India  Received June 1, 2012; accepted October 14, 2012 Summary Reciprocal crosses between Brassica carinata (BBCC, 2n=34) and B. tournefortii (TT,  ) were attempted in order to determine crossability between the species, carry out chromo- $2n{=}20^{\circ}$  somal associations among the B, C and T genomes, and to study morphology of the interspeciﬁc hy- brids. A hybrid between these 2 species was obtained only when  $B$ . carinata was used as a female  parent. Morphologically, the hybrid was intermediate between the progenitor species but its leaves  resembled  $B$ . carinata. The  $\mathrm{F}_{1}$  plant was tall but grew very slowly. The hybrid was almost male   sterile, showing only  $2.3\%$  pollen stainability. Meiotic studies of trigenomic triploid hybrid (BCT,  ) showed various chromosome conﬁgurations including quadrivalents (0–1), trivalents (0–2),  $2n{=}27.$  bivalents (2–9) and univalents (5–23). Only 5 univalents were recorded to have occurred; otherwise  trivalents and bivalents were observed in pollen mother cells of the BCT hybrid. A few chromo- somal association could be expected through allosyndesis which suggests the possibility of transfer- ring genes across the species through interspeciﬁc hybridization. 

 



Key words  Brassica carinata, Brassica tournefortii, Crossability, Cytomorphology, Interspeciﬁc  hybrid.

Ethiopian or Abyssinian mustard (Brassica carinata A. Braun,  $2n{=}34$ ) possesses many desir- able characteristics that are rare in the other Brassica oil crops: heat and drought tolerance, toler- ance to various biotic and abiotic stresses and availability of yellow-seeded germplasm (Jiang et al.  2007). Inspite of these positive attributes, the crop suffers from many agronomic limitations like  longer crop duration, poor harvest index, low oil content etc., whereas  $B$ . tournefortii  $(2n{=}20)$ ,  known variously as wild turnip or Sahara mustard or Asian mustard or Tournefortʼs mustard or  Tournefortʼs Birdrape, grown sporadically in a few pockets of arid and semi-arid areas, has a strat- egy of early and quick growth because this effectively captures available soil moisture, enables it to  build a canopy, reproduce and mature before neighbouring species (Minnich and Sanders 2000). It  has also been reported as having a good source of tolerance against aphids (Singh et al. 1965) and  drought (West and Nabhan 2002). However, lower test weight ( $\mathrm{2-3\,g}/1000$  seed) and susceptibility  to downy mildew (Rajpurohit and Choudhary 1995) are the major constraints in the spread of this  species. 

Interspeciﬁc hybridization has great potential for the improvement of Brassica. Besides the  transfer of useful traits across the species in genus Brassica (Roy 1984, Rahman 2001, Rygulla et  al. 2007), such crosses were widely used to create genetic variability (Prakash 1973a), genetic   diversity (Choudhary and Joshi 2001) and to synthesize artiﬁcially new species such as  Brassicoraphanus (Tokumasu 1970) Brassica napocampestris (McNaughton 1973) and  Raphanofortii (Choudhary et al. 2000a). Both  $B$ . carinata and  $B$ . tournefortii species have their  own advantage and deﬁciencies. Therefore, the present study was undertaken to examine crossabil- ity between  $B$ . carinata and  $B_{i}$ . tournefortii and to observe the meiotic behaviour of the interspeciﬁc   hybrid of these species as it suggests the possibility of genetic introgressions for crop improve- $\mathrm{F}_{1}$  ments which take place through chromosomal interchange. 



Materials and methods Two genotypes of  $B$ . tournefortii (TT,  $2n{=}20$ ) — RBT 58 and RBT 61 — and 1 of  $B_{i}$ . carinata  (BBCC,  $2n{=}34.$ ) — BCN selected from the germplasm collection of the Agricultural Research  Station, Mandor-Jodhpur, India were used in the hybridization programme. 



The reciprocal interspeciﬁc crosses were attempted involving the selected genotypes of both  the species under ﬁeld conditions. Emasculation and pollination were conducted as per the given  procedure. Immature ﬂower buds of appropriate size were emasculated by removal of anthers in the  evening and bagged with butter paper bags. In the following morning, stigma of these buds were  pollinated with pollen of the desired parent and the racemes were re-enclosed with bags to prevent  cross pollination. Seeds obtained from crosses and parents were grown in the ﬁeld for evaluation.  Morphological comparisons were made, and the meiosis of the interspeciﬁc hybrid was studied. 

For meiotic analysis, young ﬂower buds were ﬁxed in freshly prepared solution of propionic  acid: absolute ethanol  $(1:3)$  containing ferric chloride as a mordent. After  $24\,\mathrm{h}$  the rinsed ﬂower  buds were stored in  $70\%$  ethanol until slides were made by the acetocarmine  $(1\%)$  squash tech- nique. Chromosome associations were observed at diakinesis/metaphase I. Pollen fertility of the   hybrid and parents was estimated by pollen grain stainability in acetocarmine  $(1\%)$ .

Results Crossability The seed setting was almost double in the cross  $B$ . carinata $\times B$ . tournefortii compared to its re- ciprocal combination. Out of 112 ﬂowers buds of  $B$ . carinata pollinated with pollen of  $B_{i}$ . tourne- fortii, 22 seeds were obtained giving rise to only a single hybrid plant. On contrary, in the recipro- cal combination, out of 122 buds of  $B$ . tournefortii pollinated by  $B_{i}$ . carinata pollen, only 11 seeds  were obtained which did not yield any true hybrids. 



Morphological features of  $F_{l}$  hybrids The phenotype of the hybrid was intermediate between parent species. The size and shape of  leaves of the hybrid were closer to  $B_{i}$ . carinata. They were obovate-lanceolate, petiolate, pinnatiﬁed  to pinnatipartite with entire to slightly dentate margin, obtuse tip, sparsely hairy and dark green in  colour. The ﬂowers were medium in size with pale yellow petals but their shape was similar to  $B$ .  tournefortii. The hybrid was tall but grew very slowly and took as many as  $95\,\mathrm{d}$  to ﬂower and  $168\,\mathrm{d}$   to mature completely (Table 1). The  $\mathrm{F_{1}}$  plant exhibited enhancement in values of attributes like pri- mary and secondary branches, main raceme length, siliquae on main raceme and siliquae per plant.  On the other hand, siliqua length, seeds per siliqua, seed weight and seed yield were recorded much  lower in the hybrid compared to either of the parents. Few seeds were set in the  $\mathrm{F}_{1}$  plant due to  open pollination. However, no seed set was found under self-pollination or in backcrosses. 

Meiotic characteristics of $\boldsymbol{{F}}_{l}$  hybrids The chromosome associations observed in pollen mother cells (PMCs) of the trihaploid BCT  hybrid  $(2n{=}27)$  are given in Table 2 and some representative cells are shown in Figs. 1–3. A total of  59 PMCs of the hybrid were analyzed at diakinesis/metaphase I. At least 2 bivalents were observed  in all the PMCs, but the maximum of 9 bivalents were observed in  $17\%$  of cells (Fig. 2). Many of  them exhibited chiasmatic conﬁgurations (Fig. 2). Multivalents in the form of trivalents (0–2) and  Discussion The hybrid from cross .  $B$ carinata ${\times}B$ . tournefortii was obtained only when  $B$ . carinata was  used as a female parent. This was not anticipated as in the majority of interspeciﬁc crosses reported  in Brassica, there was greater success when the species with the higher basic chromosome number  was used as a female (Quazi 1988, Choudhary and Joshi 1999). Nishiyama et al. (1991) reported  that crosses between tetraploid and diploid species were partly successful, while the reciprocal, i.e.  diploid⊗tetraploid, usually failed. However, the occurrence of the hybrid in the direction of .  $B$  carinata⊗B. tournefortii in the present study is in contradiction to Harberdʼs theory that interspe- ciﬁc crosses with  $B$ . tournefortii were only possible when it was used as a female parent (Harberd  1976).

 



The sexual hybrid between  $B$ . carinata and  $B$ . tournefortii was reported for the ﬁrst time by us  (Joshi and Choudhary 1999). The hybrid plant was observed to be an intermediate in terms of many  phenotypic features of both the parents. This is an advantage, since it would allow better selection  for speciﬁc attributes in segregating progenies.

Meiotic studies of trigenomic triploid hybrid BCT  $(2n{=}27)$ ) revealed various chromosome con- ﬁgurations including quadrivalents (0–1), trivalents (0–2) and bivalents (2–9). Based on earlier  studies on haploid genomes of B (Prakash 1973b), C (Armstrong and Keller 1982) and T (Prakash  1974), a total of 7 bivalents were theoretically possible due to autosyndesis in these 3 genomes.  Hence, out of the maximum of 9 bivalents observed in the present hybrid, 2 could be expected to be  due to allosyndesis among the B, C and T genomes. Mizushima (1968) noticed a maximum of 3 bi- valents in a TC hybrid and opined that one of them might be due to allosyndesis between these 2  genomes, while Ljungberg et al. (1993) observed a maximum of  $1\ \mathrm{III}{+}3\ \mathrm{II}{+}10\ \mathrm{I}$  in a similar hybrid  and suggested the possibility of allosyndetic pairing in T and C genomes. Narain and Prakash  (1972) reported a maximum of 3 bivalents in a BT hybrid but did not say anything regarding the  nature of origin of such bivalents. Sarla and Raut (1991) found a maximum of 1 III+5 II+4 I in BC  hybrid and demonstrated homology between the B and C genomes. Pairing in the present triploid  hybrid has been assumed to occur mostly between the B and C genomes. However, less than 10  univalents (possibly of T genome) were counted in  $45.8\%$  of all PMCs, suggesting that some T  chromosomes of .  $B$ tournefortii might have been involved in the pairing with B and C chromo- somes.



The high level of pairing observed in the hybrid might be due to auto/allosyndesis pairing  within and among the genomes. This might be attributed to structural similarities and duplications  of some of the chromosomes (Röbbelen 1960). However, the heteromorphic appearance of biva- lents in some of the PMCs observed and the presence of only 5 univalents, instead of the  $27\ \mathrm{ex}.$ - pected, indicated the extent of homology among B, C and T genomes. Bridges at anaphase I ob- served in the present material might have resulted from pairing between partially chromosomes of  the parent genomes. Such inter-genomic pairing in Brassica hybrid was also observed by Howard  (1938). The occurrence of multivalent association at a high frequency in the hybrid studied indi- cated that parts of the chromosomes of these genomes possess sufﬁcient afﬁnity to allow multiva- lent association (Newell et al. 1984). This showed that some of chromosomes of these 3 genomes  had duplicate archetypes and conﬁrmed the hypothesis of the secondary polyploidy origin of the B,  C and T genomes. Secondly, the lower average chromosome association observed in the present  BCT hybrid, compared to that reported in an ABC hybrid by Choudhary et al. (2000b), suggested  that the T genome was more distantly related than A to both the B and C genomes. Inter-genomic  chromosome associations noticed in the hybrid indicated the possibility of transferring genes across  the species through interspeciﬁc hybridization. 

Acknowledgements The authors would like to thank the late Dr. S. Prakash, New Delhi, for his help during this in- vestigation. 



References

Armstrong, K. C. and Keller, W. A. 1982. Chromosome pairing in haploids of Brassica oleracea. Can. J. Genet. Cytol. 24:  735–739. Choudhary, B. R. and Joshi, P. 1999. Interspeciﬁc hybridization in Brassica. In: Proceedings of the  $10^{\mathrm{th}}$  International  Rapeseed Congress. Canberra, CD-ROM.  — and — 2001. Genetic diversity in advanced derivatives of Brassica interspeciﬁc hybrids. Euphytica 121: 1–7. 

—, — and Singh, K. 2000a. Synthesis, morphology and cytogenetics of Raphanofortii (TTRR, 2n=38): a new amphidiploid  of hybrid Brassica tournefortii (TT,  $2n{=}20$ )⊗Raphanus caudatus (RR,  $2n{=}18$ ). Theor. Appl. Genet. 101: 990–999. —, — and Ramarao, S. 2000b. Interspeciﬁc hybridization between Brassica carinata and B. rapa. Plant Breed. 119: 417– 420. Harberd, D. J. 1976. Cytotaxonomic studies of Brassica and related genera. In: Vaughan, J. G., MacLeod, A. J. and Jones,  B. M. G. (eds.). The Biology and Chemistry of the Cruciferae. Academic Press, London. pp. 47–68. Howard, H. W. 1938. The fertility of amphidiploids from the cross Raphanus sativum⊗Brassica oleracea. J. Genet. 36:  239–273. Jiang, Y., Tian, E., Li, R., Chen, L. and Meng, J. 2007. Genetic diversity of Brassica carinata with emphasis on the inter- speciﬁc crossability with B. rapa. Plant Breed. 126: 487–491. Joshi, P. and Choudhary, B. R., 1999. Interspeciﬁc hybridization in Brassica. I. Brassica carinata⊗B. tournefortii. In:  Proceedings of  $10^{\mathrm{th}}$  International Rapeseed Congress. Canberra, CD-ROM.  Ljungberg, A., Cheng, B. F. and Heneen, W. K. 1993. Investigation of hybrids between Brassica tournefortii Govan and   B. alboglabra Baily. Sveriges Utsadesforenings Tidskrift 103: 191–197. McNaughton, I. H. 1973. B. rassica napocampestris L ( $2n{=}58.$ ) I. Synthesis, cytology, fertility and general considerations.  

Euphytica 22: 301–309.

Minnich, R. A. and Sanders, A. C. 2000. Brassica tournefortii Gouan. In: Bossard, C. C., Randall, J. M. and Hoshovsky, M.  C. (eds.). Invasive plants of Californiaʼs wildlands. University of California Press, Berkeley and Los Angeles,  California.

Mizushima, U. 1968. Phylogenetic studies on some wild Brassica species. Tohoku Journal of Agricultural Research 19:  83–99.

Narain, A. and Prakash, S. 1972. Investigations on the artiﬁcial synthesis of amphidiploids of Brassica tournefortii Gouan  with the other elementary species of Brassica. 1. Genomic relationships. Genetica 43: 90–97.

Newell, C. A., Rhoads, M. L. and Bidney, D. L. 1984. Cytogenetic analysis of plants regenerated from tissue explants and  mesophyll protoplasts of winter rape, Brassica napus L. Can. J. Genet. Cytol. 26: 752–761.

Nishiyama, I., Sarashima, M. and Matsuzawa, Y. 1991. Critical discussion on abortive interspeciﬁc crosses in Brassica.  Plant Breed. 107: 288–302.

Prakash, S. 1973a. Non-homologous meiotic pairing in the A and B genomes of in Brassica: its breeding signiﬁcance in the  production of variable amphidiploids. Genet. Res. 21: 133–137. 

— 1973b. Haploidy in Brassica nigra Koch. Euphytica 22: 613–614. — 1974. Haploid meiosis and origin of Brassica tournefortii Gouan. Euphytica 23: 591–595. Quazi, M. H. 1988. Interspeciﬁc hybrids between Brassica napus L. and B. oleracea L. developed by embryo cul- ture. Theor. Appl. Genet. 75: 309–318.

Rahman, M. H. 2001. Production of yellow-seeded Brassica napus through interspeciﬁc crosses. Plant Breed. 120: 463– 472 Rajpurohit, T. S. and Choudhary, B. R. 1995. Downy mildew (Peronospora parasitica) of wild turnip (Brassica tournefor- tii), a new record from Rajasthan. Indian Journal of Agricultural Sciences 65: 377.

Roy, N. N. 1984. Interspeciﬁc transfer of Brassica juncea type high blackleg resistance to B. napus. Euphytica 33: 295– 303.

Röbbelen, G. 1960. Beitrage zur Analyse des Brassica Genomes. Chromosoma 11: 205–228. Rygulla, W., Snowdon, R. J., Eynck, C., Koopmann, B., von Tiedemann, A., Lühs, W. and Friedt, W. 2007. Broadening the  genetic basis of Verticillium longisporum resistance in Brassica napus by interspeciﬁc hybridization.   Phytopathology 97:1391–1396.

Sarla, N. and Raut, R. N. 1991. Cytogenetical studies on Brassica nigra $\times B$ . oleracea hybrids. Indian J Genet Plant Breed  51: 408–413.

Singh, S. R., Narain, A., Srivastava, K. P. and Siddiqui, J. A. 1965. Fecundity of the mustard aphid (Lipaphis erismi Kalt)  on different rapes and mustard species. J Oilseeds Res 9: 215–219.

Tokumasu, S. 1970. Intergeneric hybrids between Brassica japonica and Raphanus sativus. Memoirs of the College of  Agriculture Ehime University 14: 285–302.

West, P. and Nabhan, G. P. 2002. Invasive plants: their occurrence and possible impact on the central Gulf Coast of Sonara  and the Midriff Islands in the sea of Cortes. In: Tellman, B. (ed.). Invasive exotic species in the Sonoran Desert re- gion. The University of Arizona Press and the Arizona-Sonora Desert Museum, Tucson, Arizona.
# Real-time Temporal Stereo Matching using Iterative Adaptive Support Weights

J˛edrzej Kowalczuk, Eric T. Psota, and Lance C. Pérez Department of Electrical Engineering, University of Nebraska-Lincoln

### [jkowalczuk2,epsota,lperez]@unl.edu

Abstract—Stereo matching algorithms are nearly always de- signed to ﬁnd matches between a single pair of images. A method is presented that was speciﬁcally designed to operate on sequences of images. This method considers the cost of matching image points in both the spatial and temporal domain. To maintain real-time operation, a temporal cost aggregation method is used to evaluate the likelihood of matches that is invariant with respect to the number of prior images being considered. This method has been implemented on massively parallel GPU hardware, and the implementation ranks as one of the fastest and most accurate real-time stereo matching methods as measured by the Middlebury stereo performance benchmark.

I. INTRODUCTION Modern stereo matching algorithms achieve excellent results on static stereo images, as demonstrated by the Middlebury stereo performance benchmark [1], [2]. However, their ap- plication to stereo video sequences does not guarantee inter- frame consistency of matches extracted from subsequent stereo frame pairs. The lack of temporal consistency of matches between successive frames introduces spurious artifacts in the resulting disparity maps. The problem of obtaining temporally consistent sequences of disparity maps from video streams is known as the temporal stereo correspondence problem, yet the amount of research efforts oriented towards ﬁnding an effective solution to this problem is surprisingly small.



A method is proposed for real-time temporal stereo match- ing that efﬁciently propagates matching cost information be- tween consecutive frames of a stereo video sequence. This method is invariant to the number of prior frames being considered, and can be easily incorporated into any local stereo method based on edge-aware ﬁlters. The iterative adaptive support matching algorithm presented in [3] serves as a foundation for the proposed method.

disparity map. Note that individual disparities can be converted to actual depths if the geometry of the camera setup is known, i.e., the stereo conﬁguration of cameras has been pre- calibrated.

aggregate cost values within the pixel neighborhoods deﬁned by these windows. In 2005, Yoon and Kweon [4] proposed an adaptive matching cost aggregation scheme, which assigns a weight value to every pixel located in the support window of a given pixel of interest. The weight value is based on the spatial and color similarity between the pixel of interest and a pixel in its support window, and the aggregated cost is computed as a weighted average of the pixel-wise costs within the considered support window. The edge-preserving nature and matching accuracy of adaptive support weights have made them one of the most popular choices for cost aggregation in recently proposed stereo matching algorithms [3], [5]–[8].

Recently, Rheman et al. [9], [10] have revisited the cost aggregation step of stereo algorithms, and demonstrated that cost aggregation can be performed by ﬁltering of subsequent layers of the initially computed matching cost volume. In par- ticular, the edge-aware image ﬁlters, such as the bilateral ﬁlter of Tomasi and Manducci [11] or the guided ﬁlter of He [12], have been rendered useful for the problem of matching cost aggregation, enabling stereo algorithms to correctly recover disparities along object boundaries. In fact, Yoon and Kweon’s adaptive support-weight cost aggregation scheme is equivalent to the application of the so-called joint bilateral ﬁlter to the layers of the matching cost volume.

It has been demonstrated that the performance of stereo algorithms designed to match a single pair of images can be adapted to take advantage of the temporal dependencies available in stereo video sequences. Early proposed solutions to temporal stereo matching attempted to average matching costs across subsequent frames of a video sequence [13], [14]. Attempts have been made to integrate estimation of motion ﬁelds (optical ﬂow) into temporal stereo matching. The methods of [15] and [16] perform smoothing of disparities along motion vectors recovered from the video sequence. The estimation of the motion ﬁeld, however, prevents real-time implementation, since state-of-the-art optical ﬂow algorithms do not, in general, approach real-time frame rates. In a related approach, Sizintsev and Wildes [17], [18] used steerable ﬁlters to obtain descriptors characterizing motion of image features in both space and time. Unlike traditional algorithms, their method performs matching on spatio-temporal motion descriptors, rather than on pure pixel intensity values, which leads to improved temporal coherence of disparity maps at the cost of reduced accuracy at depth discontinuities.

Most recently, local stereo algorithms based on edge-aware ﬁlters were extended to incorporate temporal evidence into the matching process. The method of Richardt et al. [19] employs a variant of the bilateral grid [20] implemented on graphics hardware, which accelerates cost aggregation and allows for weighted propagation of pixel dissimilarity metrics from previous frames to the current one. Although this method outperforms the baseline frame-to-frame approach, the amount of hardware memory necessary to construct the bilateral grid limits its application to single-channel, i.e., grayscale images only. Hosni et al. [10], on the other hand, reformulated kernels of the guided image ﬁlter to operate on both spatial and temporal information, making it possible to process a temporal collection of cost volumes. The ﬁltering operation was shown to preserve spatio-temporal edges present in the cost volumes, resulting in increased temporal consistency of disparity maps, greater robustness to image noise, and more accurate behavior around object boundaries.



III. METHOD The proposed temporal stereo matching algorithm is an extension of the real-time iterative adaptive support-weight algorithm described in [3]. In addition to real-time two- pass aggregation of the cost values in the spatial domain, the proposed algorithm enhances stereo matching on video sequences by aggregating costs along the time dimension. The operation of the algorithm has been divided into four stages: 1) two-pass spatial cost aggregation, 2) temporal cost aggregation, 3) disparity selection and conﬁdence assessment, and 4) iterative disparity reﬁnement. In the following, each of these stages is described in detail.



A. Two-Pass Spatial Cost Aggregation Humans group shapes by observing the geometric distance and color similarity of points in space. To mimic this vi- sual grouping, the adaptive support-weight stereo matching algorithm [4] considers a support window  $\Omega_{p}$  centered at the pixel of interest  $p$ , and assigns a support weight to each pixel . The support weight relating pixels $q\in\Omega_{p}$  and  $q$  is given by



## 
$$
w(p,q)=\exp{\left(-\frac{\Delta_{g}(p,q)}{\gamma_{g}}-\frac{\Delta_{c}(p,q)}{\gamma_{c}}\right)},
$$


(1) where  $\Delta_{g}(p,q)$  is the geometric distance,  $\Delta_{c}(p,q)$  is the color difference between pixels  $p$  and  $q.$ , and the coefﬁcients  $\gamma_{g}$  and  regulate the strength of grouping by geometric distance and $\gamma_{c}$  color similarity, respectively.



To identify a match for the pixel of interest  $p$ , the real-time iterative adaptive support-weight algorithm evaluates matching costs between and every match candidate  $\bar{p}\in S_{p}$ , where  $S_{p}$  denotes a set of matching candidates associated with pixel  $p$ . For a pair of pixels  $p$  and  ${\bar{p}}.$ , and their support windows  $\Omega_{p}$  and  $\Omega_{\bar{p}}$ , the initial matching cost is aggregated using the matching cost by performing two-pass aggregation using two orthogonal 1D windows [5], [6], [8]. The two-pass method ﬁrst aggregates matching costs in the vertical direction, and then computes a weighted sum of the aggregated costs in the horizontal direction. Given that support regions are of size , the two-pass method reduces the complexity of cost $\omega\times\omega$  aggregation from  $\mathcal{O}(\omega^{2})$  to  $\mathcal{O}(\omega)$ .



 $B$ . Temporal cost aggregation Once aggregated costs  $C(p,\bar{p})$  have been computed for all pixels  $p$  in the reference image and their respective matching candidates  $\bar{p}$  in the target image, a single-pass temporal aggregation routine is exectuted. At each time instance, the algorithm stores an auxiliary cost  $C_{a}(p,\bar{p})$  which holds a weighted summation of costs obtained in the previous frames. During temporal aggregation, the auxiliary cost is merged with the cost obtained from the current frame using

## 
$$
C(p,\bar{p})\gets\frac{(1-\lambda)\cdot C(p,\bar{p})+\lambda\cdot w_{t}(p,p_{t-1})\cdot C_{a}(p,\bar{p})}{(1-\lambda)+\lambda\cdot w_{t}(p,p_{t-1})},
$$


 (4) where the feedback coefﬁcient  $\lambda$  controls the amount of cost smoothing and  $w_{t}(p,p_{t-1})$  enforces color similarity in the temporal domain. The temporal adaptive weight computed between the pixel of interest  $p$  in the current frame and pixel , located at the same spatial coordinate in the prior frame, $p_{t-1}$  is given by



## 
$$
w_{t}(p,p_{t-1})=\exp{\left(-\frac{\Delta_{c}(p,p_{t-1})}{\gamma_{t}}\right)},
$$


(5) where  $\gamma_{t}$  regulates the strength of grouping by color similarity in the temporal dimension. The temporal adaptive weight has the effect of preserving edges in the temporal domain, such that when a pixel coordinate transitions from one side of an edge to another in subsequent frames, the auxiliary cost is assigned a small weight and the majority of the cost is derived from the current frame.



C. Disparity Selection and Conﬁdence Assessment Having performed temporal cost aggregation, matches are determined using the Winner-Takes-All (WTA) match selec- tion criteria. The match for  $p$ , denoted as  $m(p)$ , is the can- didate pixel  $\bar{p}\in S_{p}$  characterized by the minimum matching cost, and is given by

## D. Iterative Disparity Reﬁnement

Once the ﬁrst iteration of stereo matching is complete, disparity estimates  $D_{p}^{i}$  p can be used to guide matching in subsequent iterations. This is done by penalizing disparities that deviate from their expected values. The penalty function is given by

## 
$$
\Lambda^{i}(p,\bar{p})=\alpha\times\sum_{q\in\Omega_{p}}w(p,q)F_{q}^{i-1}\left|D_{q}^{i-1}-d_{p}\right|\,,
$$


(8) where the value of  $\alpha$  is chosen empirically. Next, the penalty values are incorporated into the matching cost as



## 
$$
C^{i}(p,\bar{p})=C^{0}(p,\bar{p})+\Lambda^{i}(p,\bar{p})\,,
$$


(9) and the matches are reselected using the WTA match selection criteria. The resulting disparity maps are then post-processed using a combination of median ﬁltering and occlusion ﬁlling. Finally, the current cost becomes the auxiliary cost for the next pair of frames in the video sequence, i.e., $C_{a}(p,\bar{p})\gets C(p,\bar{p})$  for all pixels  $p$  in the and their matching candidates  $\bar{p}$ .



IV. RESULTS The speed and accuracy of real-time stereo matching al- gorithms are traditionally demonstrated using still-frame im- ages from the Middlebury stereo benchmark [1], [2]. Still frames, however, are insufﬁcient for evaluating stereo match- ing algorithms that incorporate frame-to-frame prediction to enhance matching accuracy. An alternative approach is to use a stereo video sequence with a ground truth disparity for each frame. Obtaining the ground truth disparity of real world video sequences is a difﬁcult undertaking due to the high frame rate of video and limitations in depth sensing- technology. To address the need for stereo video with ground truth disparities, ﬁve pairs of synthetic stereo video sequences of a computer-generated scene were given in [19]. While these videos incorporate a sufﬁcient amount of movement variation, they were generated from relatively simple models using low- resolution rendering, and they do not provide occlusion or discontinuity maps.



To evaluate the performance of temporal aggregation, a new synthetic stereo video sequence is introduced along with corresponding disparity maps, occlusion maps, and disconti- nuity maps for evaluating the performance of temporal stereo matching algorithms. To create the video sequence, a complex scene was constructed using Google Sketchup and a pair of animated paths were rendered photorealistically using the Kerkythea rendering software. Realistic material properties were used to give surfaces a natural-looking appearance by adjusting their specularity, reﬂectance, and diffusion. The video sequence has a resolution of  $640\,\times\,480$  pixels, a frame rate of 30 frames per second, and a duration of 4 seconds. In addition to performing photorealistic rendering, depth renders of both video sequences were also generated and converted to ground truth disparity for the stereo video. The video sequences and ground truth data have been made avail- ableat $\scriptstyle{\mathrm{http}}\,\cdot\,/\,{\mathrm{mc}}2\,.\,{\mathrm{unl}}\,.\,{\mathrm{edu}}/{\mathrm{current-research}}$  . Figure 2 shows two sample frames $/\,\mathtt{i m a g e-p r o c e s s i n g}/.$  of the synthetic stereo scene from a single camera perspective, along with the ground truth disparity, occlusion map, and discontinuity map.



Frame 30Frame 90 Figure 2: Two sample frames from the synthetic video se- quence ( $(1^{\mathrm{st}}$  row), along with their corresponding ground truth disparity  $(2^{\mathrm{nd}}$  row), occlusion map ( $(3^{\mathrm{rd}}$  row), and discontinuity map ( $(4^{\mathrm{th}}$  row).

The results of temporal stereo matching are given in Figure 3 for uniform additive noise conﬁned to the ranges of  $\pm0$ , , and $\pm20.$   $\pm40$ . Each perfo $\lambda$ mance plot is given as a function of the feedback coefﬁcient λ. As with the majority of temporal stereo matching methods, improvements are negligible when no noise is added to the images [10], [19]. This is largely due to the fact that the video used to evaluate these methods is computer generated with very little noise to start with, thus the noise suppression achieved with temporal stereo matching shows little to no improvement over methods that operate on pairs of images.

Signiﬁcant improvements in accuracy can be seen in Figure 3 when the noise has ranges of  $\pm20$ , and  $\pm40$ . In this scenario, the effect of noise in the current frame is reduced by increasing the feedback coefﬁcient  $\lambda$ . This increasing of  $\lambda$  has the effect

Noise: ±0

### 3

MSE

### 2

### 1

### 00.20.40.60.8

### λ

Noise: ±20

### 00.20.40.60.8 2 2.5 3 3.5

MSE

### λ

Noise: ±40

### 010203040 0 0.2 0.4 0.6 0.8 1

### λ

over the most recent frames. By maintaining a reasonably high value of  $\gamma_{t}$ , the auxiliary cost also preserves temporal edges, essentially reducing over-smoothing of a pixel’s dis- parity when a pixel transitions from one depth to another in subsequent frames.

the proposed implementation achieves the highest speed of operation measured by the number of disparity hypotheses evaluated per second, as shown in Table II. It is also the second most accurate real-time method in terms of error rate, as measured using the Middlebury stereo evaluation benchmark. It should be noted that it is difﬁcult to establish an unbiased metric for speed comparisons, as the architecture, number of cores, and clock speed of graphics hardware used are not consistent across implementations.

V. CONCLUSION While the majority of stereo matching algorithms focus on achieving high accuracy on still images, the volume of research aimed at recovery of temporally consistent disparity maps remains disproportionally small. This paper introduces an efﬁcient temporal cost aggregation scheme that can easily be combined with conventional spatial cost aggregation to improve the accuracy of stereo matching when operating on video sequences. A synthetic video sequence, along with ground truth disparity data, was generated to evaluate the performance of the proposed method. It was shown that temporal aggregation is signiﬁcantly more robust to noise than a method that only considers the current stereo frames.

REFERENCES [1] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” International Journal of Computer Vision, vol. 47, pp. 7–42, April-June 2002.



[2] D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light,” in In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195–202, June 2003.

[3] J. Kowalczuk, E. Psota, and L. Perez, “Real-time stereo matching on CUDA using an iterative reﬁnement method for adaptive support-weight correspondences,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 23, pp. 94 –104, Jan. 2013.

[4] K.-J. Yoon and I.-S. Kweon, “Locally adaptive support-weight approach for visual correspondence search,” in CVPR ’05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern , (Washington, DC, USA), pp. 924– 931, IEEE Computer Society, 2005.

[5] L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, “High-quality real- time stereo using adaptive cost aggregation and dynamic programming,” in 3DPVT ’06: Proceedings of the Third International Symposium , (Washington, DC, USA), pp. 798–805, IEEE Computer Society, 2006.

[6] W. Yu, T. Chen, F. Franchetti, and J. C. Hoe, “High performance stereo vision designed for massively data parallel platforms,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 20, pp. 1509 –1519, November 2010.

[7] S. Mattoccia, M. Viti, and F. Ries, “Near real-time fast bilateral stereo on the GPU,” in Computer Vision and Pattern Recognition Workshops , pp. 136 –143, June 2011.

[8] K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. Van Gool, “Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 21, pp. 867 –878, July 2011.

[9] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz, “Fast cost- volume ﬁltering for visual correspondence and beyond,” in Computer , pp. 3017 –3024, June 2011.

[10] A. Hosni, C. Rhemann, M. Bleyer, and M. Gelautz, “Temporally con- sistent disparity and optical ﬂow via efﬁcient spatio-temporal ﬁltering,” in Advances in Image and Video Technology (Y.-S. Ho, ed.), vol. 7087 of Lecture Notes in Computer Science, pp. 165–177, Springer Berlin / Heidelberg, 2012.

[11] C. Tomasi and R. Manduchi, “Bilateral ﬁltering for gray and color images,” in, pp. 839 –846, jan 1998.

[12] K. He, J. Sun, and X. Tang, “Guided image ﬁltering,” in Vision – ECCV 2010, vol. 6311 of Lecture Notes in Computer Science, pp. 1–14, Springer Berlin / Heidelberg, 2010.

[13] L. Zhang, B. Curless, and S. M. Seitz, “Spacetime stereo: Shape recovery for dynamic scenes,” in IEEE Computer Society Conference , pp. 367–374, June 2003. [14] J. Davis, D. Nehab, R. Ramamoorthi, and S. Rusinkiewicz, “Spacetime stereo: a unifying framework for depth from triangulation,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp. 296 –302, February 2005.

[15] E. Larsen, P. Mordohai, M. Pollefeys, and H. Fuchs, “Temporally consistent reconstruction from multiple video streams using enhanced belief propagation,” in , pp. 1 –8, oct. 2007. [16] M. Bleyer, M. Gelautz, C. Rother, and C. Rhemann, “A stereo approach that handles the matting problem via image warping,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference , pp. 501 –508, June 2009. [17] M. Sizintsev and R. Wildes, “Spatiotemporal stereo via spatiotemporal quadric element (stequel) matching,” in Computer Vision and Pattern , pp. 493 –500, june 2009.

[18] M. Sizintsev and R. Wildes, “Spatiotemporal stereo and scene ﬂow via stequel matching,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 34, pp. 1206 –1219, june 2012.

[19] C. Richardt, D. Orr, I. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid,” in Proceedings of the European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, pp. 510–523, September 2010.

[20] S. Paris and F. Durand, “A fast approximation of the bilateral ﬁlter using a signal processing approach,” Int. J. Comput. Vision, vol. 81, pp. 24–52, Jan. 2009.

[21] Q. Yang, L. Wang, R. Yang, S. Wang, M. Liao, and D. Nistér, “Real- time global stereo matching using hierarchical belief propagation.,” in British Machine Vision Conference, pp. 989–998, 2006.
# Real-time Temporal Stereo Matching using Iterative Adaptive Support Weights

J˛edrzej Kowalczuk, Eric T. Psota, and Lance C. Pérez Department of Electrical Engineering, University of Nebraska-Lincoln

### [jkowalczuk2,epsota,lperez]@unl.edu

Abstract—Stereo matching algorithms are nearly always de- signed to ﬁnd matches between a single pair of images. A method is presented that was speciﬁcally designed to operate on sequences of images. This method considers the cost of matching image points in both the spatial and temporal domain. To maintain real-time operation, a temporal cost aggregation method is used to evaluate the likelihood of matches that is invariant with respect to the number of prior images being considered. This method has been implemented on massively parallel GPU hardware, and the implementation ranks as one of the fastest and most accurate real-time stereo matching methods as measured by the Middlebury stereo performance benchmark.

I. INTRODUCTION Modern stereo matching algorithms achieve excellent results on static stereo images, as demonstrated by the Middlebury stereo performance benchmark [1], [2]. However, their ap- plication to stereo video sequences does not guarantee inter- frame consistency of matches extracted from subsequent stereo frame pairs. The lack of temporal consistency of matches between successive frames introduces spurious artifacts in the resulting disparity maps. The problem of obtaining temporally consistent sequences of disparity maps from video streams is known as the temporal stereo correspondence problem, yet the amount of research efforts oriented towards ﬁnding an effective solution to this problem is surprisingly small.



A method is proposed for real-time temporal stereo match- ing that efﬁciently propagates matching cost information be- tween consecutive frames of a stereo video sequence. This method is invariant to the number of prior frames being considered, and can be easily incorporated into any local stereo method based on edge-aware ﬁlters. The iterative adaptive support matching algorithm presented in [3] serves as a foundation for the proposed method.

disparity map. Note that individual disparities can be converted to actual depths if the geometry of the camera setup is known, i.e., the stereo conﬁguration of cameras has been pre- calibrated.

aggregate cost values within the pixel neighborhoods deﬁned by these windows. In 2005, Yoon and Kweon [4] proposed an adaptive matching cost aggregation scheme, which assigns a weight value to every pixel located in the support window of a given pixel of interest. The weight value is based on the spatial and color similarity between the pixel of interest and a pixel in its support window, and the aggregated cost is computed as a weighted average of the pixel-wise costs within the considered support window. The edge-preserving nature and matching accuracy of adaptive support weights have made them one of the most popular choices for cost aggregation in recently proposed stereo matching algorithms [3], [5]–[8].

Recently, Rheman et al. [9], [10] have revisited the cost aggregation step of stereo algorithms, and demonstrated that cost aggregation can be performed by ﬁltering of subsequent layers of the initially computed matching cost volume. In par- ticular, the edge-aware image ﬁlters, such as the bilateral ﬁlter of Tomasi and Manducci [11] or the guided ﬁlter of He [12], have been rendered useful for the problem of matching cost aggregation, enabling stereo algorithms to correctly recover disparities along object boundaries. In fact, Yoon and Kweon’s adaptive support-weight cost aggregation scheme is equivalent to the application of the so-called joint bilateral ﬁlter to the layers of the matching cost volume.

It has been demonstrated that the performance of stereo algorithms designed to match a single pair of images can be adapted to take advantage of the temporal dependencies available in stereo video sequences. Early proposed solutions to temporal stereo matching attempted to average matching costs across subsequent frames of a video sequence [13], [14]. Attempts have been made to integrate estimation of motion ﬁelds (optical ﬂow) into temporal stereo matching. The methods of [15] and [16] perform smoothing of disparities along motion vectors recovered from the video sequence. The estimation of the motion ﬁeld, however, prevents real-time implementation, since state-of-the-art optical ﬂow algorithms do not, in general, approach real-time frame rates. In a related approach, Sizintsev and Wildes [17], [18] used steerable ﬁlters to obtain descriptors characterizing motion of image features in both space and time. Unlike traditional algorithms, their method performs matching on spatio-temporal motion descriptors, rather than on pure pixel intensity values, which leads to improved temporal coherence of disparity maps at the cost of reduced accuracy at depth discontinuities.

Most recently, local stereo algorithms based on edge-aware ﬁlters were extended to incorporate temporal evidence into the matching process. The method of Richardt et al. [19] employs a variant of the bilateral grid [20] implemented on graphics hardware, which accelerates cost aggregation and allows for weighted propagation of pixel dissimilarity metrics from previous frames to the current one. Although this method outperforms the baseline frame-to-frame approach, the amount of hardware memory necessary to construct the bilateral grid limits its application to single-channel, i.e., grayscale images only. Hosni et al. [10], on the other hand, reformulated kernels of the guided image ﬁlter to operate on both spatial and temporal information, making it possible to process a temporal collection of cost volumes. The ﬁltering operation was shown to preserve spatio-temporal edges present in the cost volumes, resulting in increased temporal consistency of disparity maps, greater robustness to image noise, and more accurate behavior around object boundaries.



III. METHOD The proposed temporal stereo matching algorithm is an extension of the real-time iterative adaptive support-weight algorithm described in [3]. In addition to real-time two- pass aggregation of the cost values in the spatial domain, the proposed algorithm enhances stereo matching on video sequences by aggregating costs along the time dimension. The operation of the algorithm has been divided into four stages: 1) two-pass spatial cost aggregation, 2) temporal cost aggregation, 3) disparity selection and conﬁdence assessment, and 4) iterative disparity reﬁnement. In the following, each of these stages is described in detail.



A. Two-Pass Spatial Cost Aggregation Humans group shapes by observing the geometric distance and color similarity of points in space. To mimic this vi- sual grouping, the adaptive support-weight stereo matching algorithm [4] considers a support window  $\Omega_{p}$  centered at the pixel of interest  $p$ , and assigns a support weight to each pixel . The support weight relating pixels $q\in\Omega_{p}$  and  $q$  is given by



## 
$$
w(p,q)=\exp{\left(-\frac{\Delta_{g}(p,q)}{\gamma_{g}}-\frac{\Delta_{c}(p,q)}{\gamma_{c}}\right)},
$$


(1) where  $\Delta_{g}(p,q)$  is the geometric distance,  $\Delta_{c}(p,q)$  is the color difference between pixels  $p$  and  $q.$ , and the coefﬁcients  $\gamma_{g}$  and  regulate the strength of grouping by geometric distance and $\gamma_{c}$  color similarity, respectively.



To identify a match for the pixel of interest  $p$ , the real-time iterative adaptive support-weight algorithm evaluates matching costs between and every match candidate  $\bar{p}\in S_{p}$ , where  $S_{p}$  denotes a set of matching candidates associated with pixel  $p$ . For a pair of pixels  $p$  and  ${\bar{p}}.$ , and their support windows  $\Omega_{p}$  and  $\Omega_{\bar{p}}$ , the initial matching cost is aggregated using the matching cost by performing two-pass aggregation using two orthogonal 1D windows [5], [6], [8]. The two-pass method ﬁrst aggregates matching costs in the vertical direction, and then computes a weighted sum of the aggregated costs in the horizontal direction. Given that support regions are of size , the two-pass method reduces the complexity of cost $\omega\times\omega$  aggregation from  $\mathcal{O}(\omega^{2})$  to  $\mathcal{O}(\omega)$ .



 $B$ . Temporal cost aggregation Once aggregated costs  $C(p,\bar{p})$  have been computed for all pixels  $p$  in the reference image and their respective matching candidates  $\bar{p}$  in the target image, a single-pass temporal aggregation routine is exectuted. At each time instance, the algorithm stores an auxiliary cost  $C_{a}(p,\bar{p})$  which holds a weighted summation of costs obtained in the previous frames. During temporal aggregation, the auxiliary cost is merged with the cost obtained from the current frame using

## 
$$
C(p,\bar{p})\gets\frac{(1-\lambda)\cdot C(p,\bar{p})+\lambda\cdot w_{t}(p,p_{t-1})\cdot C_{a}(p,\bar{p})}{(1-\lambda)+\lambda\cdot w_{t}(p,p_{t-1})},
$$


 (4) where the feedback coefﬁcient  $\lambda$  controls the amount of cost smoothing and  $w_{t}(p,p_{t-1})$  enforces color similarity in the temporal domain. The temporal adaptive weight computed between the pixel of interest  $p$  in the current frame and pixel , located at the same spatial coordinate in the prior frame, $p_{t-1}$  is given by



## 
$$
w_{t}(p,p_{t-1})=\exp{\left(-\frac{\Delta_{c}(p,p_{t-1})}{\gamma_{t}}\right)},
$$


(5) where  $\gamma_{t}$  regulates the strength of grouping by color similarity in the temporal dimension. The temporal adaptive weight has the effect of preserving edges in the temporal domain, such that when a pixel coordinate transitions from one side of an edge to another in subsequent frames, the auxiliary cost is assigned a small weight and the majority of the cost is derived from the current frame.



C. Disparity Selection and Conﬁdence Assessment Having performed temporal cost aggregation, matches are determined using the Winner-Takes-All (WTA) match selec- tion criteria. The match for  $p$ , denoted as  $m(p)$ , is the can- didate pixel  $\bar{p}\in S_{p}$  characterized by the minimum matching cost, and is given by

## D. Iterative Disparity Reﬁnement

Once the ﬁrst iteration of stereo matching is complete, disparity estimates  $D_{p}^{i}$  p can be used to guide matching in subsequent iterations. This is done by penalizing disparities that deviate from their expected values. The penalty function is given by

## 
$$
\Lambda^{i}(p,\bar{p})=\alpha\times\sum_{q\in\Omega_{p}}w(p,q)F_{q}^{i-1}\left|D_{q}^{i-1}-d_{p}\right|\,,
$$


(8) where the value of  $\alpha$  is chosen empirically. Next, the penalty values are incorporated into the matching cost as



## 
$$
C^{i}(p,\bar{p})=C^{0}(p,\bar{p})+\Lambda^{i}(p,\bar{p})\,,
$$


(9) and the matches are reselected using the WTA match selection criteria. The resulting disparity maps are then post-processed using a combination of median ﬁltering and occlusion ﬁlling. Finally, the current cost becomes the auxiliary cost for the next pair of frames in the video sequence, i.e., $C_{a}(p,\bar{p})\gets C(p,\bar{p})$  for all pixels  $p$  in the and their matching candidates  $\bar{p}$ .



IV. RESULTS The speed and accuracy of real-time stereo matching al- gorithms are traditionally demonstrated using still-frame im- ages from the Middlebury stereo benchmark [1], [2]. Still frames, however, are insufﬁcient for evaluating stereo match- ing algorithms that incorporate frame-to-frame prediction to enhance matching accuracy. An alternative approach is to use a stereo video sequence with a ground truth disparity for each frame. Obtaining the ground truth disparity of real world video sequences is a difﬁcult undertaking due to the high frame rate of video and limitations in depth sensing- technology. To address the need for stereo video with ground truth disparities, ﬁve pairs of synthetic stereo video sequences of a computer-generated scene were given in [19]. While these videos incorporate a sufﬁcient amount of movement variation, they were generated from relatively simple models using low- resolution rendering, and they do not provide occlusion or discontinuity maps.



To evaluate the performance of temporal aggregation, a new synthetic stereo video sequence is introduced along with corresponding disparity maps, occlusion maps, and disconti- nuity maps for evaluating the performance of temporal stereo matching algorithms. To create the video sequence, a complex scene was constructed using Google Sketchup and a pair of animated paths were rendered photorealistically using the Kerkythea rendering software. Realistic material properties were used to give surfaces a natural-looking appearance by adjusting their specularity, reﬂectance, and diffusion. The video sequence has a resolution of  $640\,\times\,480$  pixels, a frame rate of 30 frames per second, and a duration of 4 seconds. In addition to performing photorealistic rendering, depth renders of both video sequences were also generated and converted to ground truth disparity for the stereo video. The video sequences and ground truth data have been made avail- ableat $\scriptstyle{\mathrm{http}}\,\cdot\,/\,{\mathrm{mc}}2\,.\,{\mathrm{unl}}\,.\,{\mathrm{edu}}/{\mathrm{current-research}}$  . Figure 2 shows two sample frames $/\,\mathtt{i m a g e-p r o c e s s i n g}/.$  of the synthetic stereo scene from a single camera perspective, along with the ground truth disparity, occlusion map, and discontinuity map.



Frame 30Frame 90 Figure 2: Two sample frames from the synthetic video se- quence ( $(1^{\mathrm{st}}$  row), along with their corresponding ground truth disparity  $(2^{\mathrm{nd}}$  row), occlusion map ( $(3^{\mathrm{rd}}$  row), and discontinuity map ( $(4^{\mathrm{th}}$  row).

The results of temporal stereo matching are given in Figure 3 for uniform additive noise conﬁned to the ranges of  $\pm0$ , , and $\pm20.$   $\pm40$ . Each perfo $\lambda$ mance plot is given as a function of the feedback coefﬁcient λ. As with the majority of temporal stereo matching methods, improvements are negligible when no noise is added to the images [10], [19]. This is largely due to the fact that the video used to evaluate these methods is computer generated with very little noise to start with, thus the noise suppression achieved with temporal stereo matching shows little to no improvement over methods that operate on pairs of images.

Signiﬁcant improvements in accuracy can be seen in Figure 3 when the noise has ranges of  $\pm20$ , and  $\pm40$ . In this scenario, the effect of noise in the current frame is reduced by increasing the feedback coefﬁcient  $\lambda$ . This increasing of  $\lambda$  has the effect

Noise: ±0

### 3

MSE

### 2

### 1

### 00.20.40.60.8

### λ

Noise: ±20

### 00.20.40.60.8 2 2.5 3 3.5

MSE

### λ

Noise: ±40

### 010203040 0 0.2 0.4 0.6 0.8 1

### λ

over the most recent frames. By maintaining a reasonably high value of  $\gamma_{t}$ , the auxiliary cost also preserves temporal edges, essentially reducing over-smoothing of a pixel’s dis- parity when a pixel transitions from one depth to another in subsequent frames.

the proposed implementation achieves the highest speed of operation measured by the number of disparity hypotheses evaluated per second, as shown in Table II. It is also the second most accurate real-time method in terms of error rate, as measured using the Middlebury stereo evaluation benchmark. It should be noted that it is difﬁcult to establish an unbiased metric for speed comparisons, as the architecture, number of cores, and clock speed of graphics hardware used are not consistent across implementations.

V. CONCLUSION While the majority of stereo matching algorithms focus on achieving high accuracy on still images, the volume of research aimed at recovery of temporally consistent disparity maps remains disproportionally small. This paper introduces an efﬁcient temporal cost aggregation scheme that can easily be combined with conventional spatial cost aggregation to improve the accuracy of stereo matching when operating on video sequences. A synthetic video sequence, along with ground truth disparity data, was generated to evaluate the performance of the proposed method. It was shown that temporal aggregation is signiﬁcantly more robust to noise than a method that only considers the current stereo frames.

REFERENCES [1] D. Scharstein and R. Szeliski, “A taxonomy and evaluation of dense two-frame stereo correspondence algorithms,” International Journal of Computer Vision, vol. 47, pp. 7–42, April-June 2002.



[2] D. Scharstein and R. Szeliski, “High-accuracy stereo depth maps using structured light,” in In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 195–202, June 2003.

[3] J. Kowalczuk, E. Psota, and L. Perez, “Real-time stereo matching on CUDA using an iterative reﬁnement method for adaptive support-weight correspondences,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 23, pp. 94 –104, Jan. 2013.

[4] K.-J. Yoon and I.-S. Kweon, “Locally adaptive support-weight approach for visual correspondence search,” in CVPR ’05: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern , (Washington, DC, USA), pp. 924– 931, IEEE Computer Society, 2005.

[5] L. Wang, M. Liao, M. Gong, R. Yang, and D. Nister, “High-quality real- time stereo using adaptive cost aggregation and dynamic programming,” in 3DPVT ’06: Proceedings of the Third International Symposium , (Washington, DC, USA), pp. 798–805, IEEE Computer Society, 2006.

[6] W. Yu, T. Chen, F. Franchetti, and J. C. Hoe, “High performance stereo vision designed for massively data parallel platforms,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 20, pp. 1509 –1519, November 2010.

[7] S. Mattoccia, M. Viti, and F. Ries, “Near real-time fast bilateral stereo on the GPU,” in Computer Vision and Pattern Recognition Workshops , pp. 136 –143, June 2011.

[8] K. Zhang, J. Lu, Q. Yang, G. Lafruit, R. Lauwereins, and L. Van Gool, “Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. 21, pp. 867 –878, July 2011.

[9] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, and M. Gelautz, “Fast cost- volume ﬁltering for visual correspondence and beyond,” in Computer , pp. 3017 –3024, June 2011.

[10] A. Hosni, C. Rhemann, M. Bleyer, and M. Gelautz, “Temporally con- sistent disparity and optical ﬂow via efﬁcient spatio-temporal ﬁltering,” in Advances in Image and Video Technology (Y.-S. Ho, ed.), vol. 7087 of Lecture Notes in Computer Science, pp. 165–177, Springer Berlin / Heidelberg, 2012.

[11] C. Tomasi and R. Manduchi, “Bilateral ﬁltering for gray and color images,” in, pp. 839 –846, jan 1998.

[12] K. He, J. Sun, and X. Tang, “Guided image ﬁltering,” in Vision – ECCV 2010, vol. 6311 of Lecture Notes in Computer Science, pp. 1–14, Springer Berlin / Heidelberg, 2010.

[13] L. Zhang, B. Curless, and S. M. Seitz, “Spacetime stereo: Shape recovery for dynamic scenes,” in IEEE Computer Society Conference , pp. 367–374, June 2003. [14] J. Davis, D. Nehab, R. Ramamoorthi, and S. Rusinkiewicz, “Spacetime stereo: a unifying framework for depth from triangulation,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 27, pp. 296 –302, February 2005.

[15] E. Larsen, P. Mordohai, M. Pollefeys, and H. Fuchs, “Temporally consistent reconstruction from multiple video streams using enhanced belief propagation,” in , pp. 1 –8, oct. 2007. [16] M. Bleyer, M. Gelautz, C. Rother, and C. Rhemann, “A stereo approach that handles the matting problem via image warping,” in Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference , pp. 501 –508, June 2009. [17] M. Sizintsev and R. Wildes, “Spatiotemporal stereo via spatiotemporal quadric element (stequel) matching,” in Computer Vision and Pattern , pp. 493 –500, june 2009.

[18] M. Sizintsev and R. Wildes, “Spatiotemporal stereo and scene ﬂow via stequel matching,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 34, pp. 1206 –1219, june 2012.

[19] C. Richardt, D. Orr, I. Davies, A. Criminisi, and N. A. Dodgson, “Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid,” in Proceedings of the European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, pp. 510–523, September 2010.

[20] S. Paris and F. Durand, “A fast approximation of the bilateral ﬁlter using a signal processing approach,” Int. J. Comput. Vision, vol. 81, pp. 24–52, Jan. 2009.

[21] Q. Yang, L. Wang, R. Yang, S. Wang, M. Liao, and D. Nistér, “Real- time global stereo matching using hierarchical belief propagation.,” in British Machine Vision Conference, pp. 989–998, 2006.
Committee Opinion No. 723: Guidelines for Diagnostic Imaging During Pregnancy and Lactation: Correction In “Committee Opinion No. 723: Guidelines for Diagnostic Imaging During Pregnancy and Lacta- tion” from the American College of Obstetricians and Gynecologists, there is an error on page e211 in the sentence on line 13 of the “Ultrasonogra- phy” section of the document. The text incorrectly reads,  $\mathrm{\mathrm{42^{\circ}C\left(35.6^{\circ}F\right)^{\mathrm{\prime}}}}$  and should instead read,  $^{\circ}\mathrm{{2}^{\circ}\dot{C}}$   $(3.6^{\circ}\mathrm{F})$ .” The full, corrected sentence is as follows: “At this intensity, the theoretical increase in tem- perature elevation for the fetus may be as high as  $\mathrm{2^{\circ}C\ (3.6^{\circ}F)}$  (2, 3).”

## REFERENCE

1. Guidelines for diagnostic imaging during pregnancy and lacta- tion. Committee Opinion No. 723. American College of Obste- tricians and Gynecologists. Obstet Gynecol 2017;130:e210–6.



### DOI: 10.1097/AOG.0000000000002858

# Committee Opinion No. 738: Aromatase Inhibitors in Gynecologic Practice: Correction

In “Committee Opinion No. 738: Aromatase Inhibitors in Gynecologic Practice” from the American College of Obstetricians and Gyne- cologists, there is an error on page e196, sec- ond column, first paragraph. In the first full sentence, the phrase “letrozole was more effec- tive than clomiphene citrate with a higher live birth rate  $(27.{\hat{5}}\%$  versus  $10.1\%$ .  $\scriptstyle{P={\stackrel{\smile}{007}}},$  is incorrect and should instead read, “letrozole was more effective than clomiphene citrate with a higher live birth rate  $(27.\bar{5}\%$  versus  $19.1\%$ , . $\scriptstyle P=.{\breve{0}}07)$  The full, corrected sentence is as fol- lows: “However, in a more recent randomized controlled trial, letrozole was more effective than clomiphene citrate with a higher live birth rate  $(27.5\%$  versus  $19.1\%,\,P{=}.00\bar{7})$ , and cumu- lativeovulationrate $(61.7\%$ versus $48.3\%$ ,

##  $\scriptstyle{P<.001}.$ 

## Reference

Aromatase inhibitors in gynecologic practice. ACOG Committee Opinion No. 738. American College of Obstetricians and Gynecol- ogists. Obstet Gynecol 2018;131:e194–9.

### DOI: 10.1097/AOG.0000000000002879

Effects of the Affordable Care Act on Young Women With Gynecologic Cancers: Correction There is an error in a range of years reported in the abstract of the article, “Effects of the Affordable Care Act on Young Women With Gynecologic Cancers: Correction.”1 In the abstract, Methods section, the incorrect sentence is, “We used the National Cancer Database with the 2004–2009 sur- veys as the pre-ACA years and the 2011–2014 sur- veys as the post-ACA years.” The correct sentence is, “We used the National Cancer Database with the 2006–2009 surveys as the pre-ACA years and the 2011–2014 surveys as the post-ACA years.”

## Reference

Smith AJB, Fader AN. Effects of the Affordable Care Act on young women with gynecologic cancers. Obstet Gynecol 2018;131:966–76.

### DOI: 10.1097/AOG.0000000000002880
Committee Opinion No. 723: Guidelines for Diagnostic Imaging During Pregnancy and Lactation: Correction In “Committee Opinion No. 723: Guidelines for Diagnostic Imaging During Pregnancy and Lacta- tion” from the American College of Obstetricians and Gynecologists, there is an error on page e211 in the sentence on line 13 of the “Ultrasonogra- phy” section of the document. The text incorrectly reads,  $\mathrm{\mathrm{42^{\circ}C\left(35.6^{\circ}F\right)^{\mathrm{\prime}}}}$  and should instead read,  $^{\circ}\mathrm{{2}^{\circ}\dot{C}}$   $(3.6^{\circ}\mathrm{F})$ .” The full, corrected sentence is as follows: “At this intensity, the theoretical increase in tem- perature elevation for the fetus may be as high as  $\mathrm{2^{\circ}C\ (3.6^{\circ}F)}$  (2, 3).”

## REFERENCE

1. Guidelines for diagnostic imaging during pregnancy and lacta- tion. Committee Opinion No. 723. American College of Obste- tricians and Gynecologists. Obstet Gynecol 2017;130:e210–6.



### DOI: 10.1097/AOG.0000000000002858

# Committee Opinion No. 738: Aromatase Inhibitors in Gynecologic Practice: Correction

In “Committee Opinion No. 738: Aromatase Inhibitors in Gynecologic Practice” from the American College of Obstetricians and Gyne- cologists, there is an error on page e196, sec- ond column, first paragraph. In the first full sentence, the phrase “letrozole was more effec- tive than clomiphene citrate with a higher live birth rate  $(27.{\hat{5}}\%$  versus  $10.1\%$ .  $\scriptstyle{P={\stackrel{\smile}{007}}},$  is incorrect and should instead read, “letrozole was more effective than clomiphene citrate with a higher live birth rate  $(27.\bar{5}\%$  versus  $19.1\%$ , . $\scriptstyle P=.{\breve{0}}07)$  The full, corrected sentence is as fol- lows: “However, in a more recent randomized controlled trial, letrozole was more effective than clomiphene citrate with a higher live birth rate  $(27.5\%$  versus  $19.1\%,\,P{=}.00\bar{7})$ , and cumu- lativeovulationrate $(61.7\%$ versus $48.3\%$ ,

##  $\scriptstyle{P<.001}.$ 

## Reference

Aromatase inhibitors in gynecologic practice. ACOG Committee Opinion No. 738. American College of Obstetricians and Gynecol- ogists. Obstet Gynecol 2018;131:e194–9.

### DOI: 10.1097/AOG.0000000000002879

Effects of the Affordable Care Act on Young Women With Gynecologic Cancers: Correction There is an error in a range of years reported in the abstract of the article, “Effects of the Affordable Care Act on Young Women With Gynecologic Cancers: Correction.”1 In the abstract, Methods section, the incorrect sentence is, “We used the National Cancer Database with the 2004–2009 sur- veys as the pre-ACA years and the 2011–2014 sur- veys as the post-ACA years.” The correct sentence is, “We used the National Cancer Database with the 2006–2009 surveys as the pre-ACA years and the 2011–2014 surveys as the post-ACA years.”

## Reference

Smith AJB, Fader AN. Effects of the Affordable Care Act on young women with gynecologic cancers. Obstet Gynecol 2018;131:966–76.

### DOI: 10.1097/AOG.0000000000002880
# Review The Exploration of Natural Compounds for Anti-Diabetes from Distinctive Species Garcinia linii with Comprehensive Review of the Garcinia Family

Ting-Hsu Chen , May-Jywan Tsai , Yaw-Syan  $\mathbf{Fu}^{3,4}$  and Ching-Feng Weng \*

1 Department of Life Science and Institute of Biotechnology, National Dong Hwa University, Hualien 97401, Taiwan; 410613031@gms.ndhu.edu.tw 2 Neural Regeneration Laboratory, Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei 11221, Taiwan; mjtsai2@vghtpe.gov.tw 3 Department of Biomedical Science and Environmental Biology, Kaohsiung Medical University, Kaohsiung city 80708, Taiwan; m805004@kmu.edu.tw 4 Institute of Respiratory Disease, Department of Basic Medical Science, Xiamen Medical College, Xiamen 361023, China \*Correspondence: cfweng-cfweng@hotmail.com; Tel.: +886-3-890-3609; Fax: +886-3-890-0163 Received: 17 September 2019; Accepted: 21 October 2019; Published: 23 October 2019



 

Abstract: Approximately 400 Garcinia species are distributed around the world. Previous studies have reported the extracts from bark, seed, fruits, peels, leaves, and stems of Garcinia mangostana, G. xanthochymus G. cambogia cardiovascular diseases, and diabetes. Moreover, the hypoglycemic eﬀects and underlined actions of diﬀerent species such as G. kola G. pedunculata G. prainiana the anti-hyperglycemia of  $G$ . linii remains to be veriﬁed in this aspect. In this article, the published literature was collected and reviewed based on the medicinal characteristics of the species Garcinia, particularly in diabetic care to deliberate the known constituents from Garcinia and further focus on and isolate new compounds of G. linii (Taiwan distinctive species) on various hypoglycemic targets including  $\alpha$ -amylase,  $\alpha$ -glucosidase,  $5^{\prime}.$ -adenosine monophosphate-activated protein kinase (AMPK), insulin receptor kinase, peroxisome proliferator-activated receptor gamma (PPARγ), and dipeptidyl peptidase-4 (DPP-4) via the molecular docking approach with Gold program to explore the potential candidates for anti-diabetic treatments. Accordingly, benzopyrans and triterpenes are postulated to be the active components in G. linii for mediating blood glucose. To further validate the potency of those active components, in vitro enzymatic and cellular function assays with in vivo animal eﬃcacy experiments need to be performed in the near future.

Keywords: Garcinia linii; hypoglycemia; benzopyran; triterpene; bioﬂavonoid; phenolic; in silico

## 1. Introduction

## 1.1. Impact of Diabetes

Diabetes mellitus (DM) is a global health issue due to its high risk factors, e.g., obesity, physical inactivity, ageing, bad eating habits, genetic predisposition, hypertension, and hyperlipidemia [1]. It is worth noting that DM is a metabolic disease and more than 400 million people suﬀered from diabetes in 2014 [2]. Interestingly, the adult diabetic population of 2014 has risen from  $4.7\%$  to  $8.5\%$  worldwide in contrast to the population in 1980 [2]; the morbidity and mortality were  $4.95\%$  and  $4.00\%,$  respectively [3]. In the future, the diabetic population will increase to 642 million by around 2045 and this population will continue to grow [3].

## 1.2. Therapy Agent of Diabetes

Nowadays, many clinical medicines such as  $\alpha\cdot$ -glucosidase and  $\alpha$ -amylase inhibitors such as Acarbose are applied to delay the metabolism of carbohydrates and control post-meal blood glucose for diabetes patients [4]. Metformin and 5-Amino-4-Imidazolecarboxamide Riboside (AICAR) are activators that increase the glucose absorption of skeletal muscles and inhibit gluconeogenesis of the liver [5,6]. The major anti-diabetes functions of Sitagliptin increase concentrations of incretin. Incretin can further potentiate the pancreas to produce insulin and inhibit the production of glucagon to decrease blood glucose levels [7]. Rosiglitazone can reduce the resistance of insulin absorption in liver cells, skeletal muscle cells, and fatty tissues [8]. Additionally, GW-9662 is an agonist that promotes peroxisome proliferator-activated receptors gamma (PPARγ) expression, which could stimulate fat metabolism, and trigger insulin pathways to regulate blood glucose and has anti-inﬂammatory properties [9].

## 1.3. Distribution of Garcinia Plants and Recent Discovery of Anti-Diabetic Agents with Garcinia Plants

Approximately 400 Garcinia species are distributed around the world including Bangladesh, China, India, Indonesia, Taiwan, Thailand, tropical Asia, Southern Africa, and Western Polynesia. Usually, the Garcinia plants are shrubs or trees. Most of their edible fruits are used in agricultural societies or the fruits and seeds are used to produce oils and dyes, and to treat various diseases, e.g., abdominal pain, food allergies, arthritis, diarrhea, dysentery, and wound infections as past research has shown [10–16 Garcinia subelliptica used as building materials to prevent destruction by typhoons in ancient Japan, which is in contrast to  G. subelliptica17,]. Moreover,   $G$ . mangostana, G. xanthochymus G. cambogia for calories or nutrition and use the extracts of G. xanthochymus G. cambogia increase the sour ﬂavor in India. Interestingly, the extract of G. cambogia preserving food freshness [18,19].

The accumulated literature showed that metabolic syndromes gradually became public health problems such as obesity, hyperglycemia, hyperlipidemia, and hypertension and lead to cardiovascular diseases (i.e., atherosclerosis, stroke, or peripheral artery disease) or diabetes [20]. Hereafter, the medicinal plants or herbs have important characteristics that resist the threat of these diseases because the adverse side eﬀects of natural compounds isolated from medicinal plants are reduced and often less severe than those from clinical drugs [21]. Interestingly, previous studies have reported that the   $G$ . mangostana, G. xanthochymus G. cambogia diabetes. Furthermore, these extracts could also trigger the myotubes and skeletal cells to absorb glucose and to balance blood glucose levels [20,22–26]. Remarkably, numerous studies have indicated that the Garcinia species, e.g., G. cambogia10 G. xanthochymus11 G. kola12 G. mangostana13,], G. pedunculata15 G. prainiana16] contain plenty of biﬂavonoids and phenolic compounds. These compounds have been found to inhibit the enzymatic activity of  $\alpha$ -amylase and  $\alpha$ -glucosidase for propagating an anti-diabetic eﬀect [27]. Concurrently, the variability of biological actions includes anti-diabetic agents that are dependent on the constituents of plants that grow locality, diﬀerently used parts (root, leaves, ﬂowers), or seasonal harvests that exhibit various compositions or ratios for each component, metabolite, or derivative including diﬀerent delivery systems such as powers, pills, or teas, among others. Generally, batch to batch control or a standard operating system via index chemicals or ﬁngerprints is crucial for the evaluation of the eﬃcacy of crude extracts, the yields in isolation, and the puriﬁcation of active ingredients. In this article, the published articles were collected and reviewed. We have addressed the isolated compounds for all Garcinia species by molecular structure and brieﬂy described their targeted biomolecules for anti-diabetic function according to chemical category via PubChem such as benzophenone (one compound isolated from G. mangostana G. mangostana  G. kola G. mangostana compounds isolated from G. xanthochymus G. hanburyi  G. mangostana species for the care of DM (Table 1) are listed to deliberate on the potential constituents of certain Garcinia species. Interestingly, G. linii is one endemic evergreen tree only distributed in outlying islands—Langyu land and Green island of Taiwan. To the best of our knowledge, active constituents including 15 xanthones, 6 biphenyls, 2 benzopyran, and 13 known compounds isolated from the root of  $G$ . linii have been reported with anti-tubercular activity and cytotoxicity [28,]. However, the medicinal values of G. linii extract as an anti-diabetic agent remain to be explored. Moreover, this review article simultaneously oﬀers insights into dissecting the molecular mechanism of isolated compounds such as the three new xanthones (linixanthones A–C), ﬁve new biphenyls (garcibiphenyls A–E), and two new benzopyran (garcibenzopyran and (S)-3-hydroxygarcibenzopyran) of the G. linii root from Taiwan [28,O-Methylmacluraxanthone;  $^{1,5}\!.$ -Dihydroxyxanthone; 1,6-Dihydroxy-3,5,7-trimethoxyxanthone; 1,6-Dihydroxy-5-methoxyxanthone; 1,6-dihydroxy- $^{5,7}\!.$ -dimethoxyxanthone; 1,6-Dihydroxy-7-methoxyxanthone; 1,7-Dihydroxyxanthone; 1,7-Dihydroxy-3-methoxyxanthone;5-Hydroxy-1-methoxyxanthone)fromtheGarcinia family on various hypoglycemic targets including  $\alpha$ -amylase,-glucosidase, $\propto$ -adenosine $5^{\prime}.$  monophosphate-activated protein kinase (AMPK), insulin receptor kinase (IRK), PPARγ, and dipeptidyl peptidase-4 (DPP4) via the molecular docking approach with Gold program (Cambridge Crystallographic Data Centre, Cambridge, UK).



## 1.4. α-Amylase and  $\alpha$ -Glucosidase

To regulate the postprandial blood glucose level, diabetic patients took carbohydrate hydrolase inhibitors such as  $\alpha$ -glucosidase and  $\alpha$ -amylase to avoid hyperglycemia.  $\alpha$ -amylase and  $\alpha$ -glucosidase are the key enzymes to hydrolyze carbohydrates and help glucose ingestion [30]. Therefore, diabetic patients have to control their blood glucose by using clinical drugs such as  $P r e c o s e^{\circled{R}}$  (Acarbose) and  $G l y s e t^{\widehat{\mathtt{Q}}}$  (Miglitol) [4] or other anti-diabetic natural compounds [27] to prolong hydrolysis of carbohydrates against hyperglycemia. In cumulative studies, a few crude extracts from the Garcinia species, e.g., G. cambogia G. xanthochymus G. kola G. mangostana10–14], containing biﬂavonoids, polyphenols, and xanthones, also inhibit the enzyme activity of  $\alpha$ -amylase and  $\propto$ -glucosidase. Therefore, the extracts are able to help diabetic patients to control their blood glucose levels by the inhibition of carbohydrate hydrolysis. Our docking results (Figure 1) showed that benzopyrans and triterpenes had a higher binding aﬃnity with  $\alpha$ -amylase and  $\alpha$ -glucosidase than with biﬂavonoid and phenolic compounds. Additionally,  $\alpha$ -tocopherolquinone (a kind of benzopyrans) and squalene (a kind of triterpenes) had a high binding aﬃnity with  $\alpha$ -amylase and  $\alpha$ -glucosidase to prolong carbohydrate hydrolyzation, reduce the absorption of glucose and mediate the blood glucose level.

## 1.5.  $5^{\prime}$ -Adenosine Monophosphate-Activated Protein Kinase (AMPK)

The  $5^{\prime}$ -adenosine monophosphate-activated protein kinase (AMPK) is composed of  $\alpha$ subunits, regulatory  $\upbeta$  subunits, and r subunits and is a sensor of cellular energy level. Cellular energy levels were changed by the ratio of AMP:ATP and ADP:ATP that inﬂuenced cellular growth and survival [31]. Previous research indicated that AMPK was activated and the rate of ATP-generating processes would increase while the rate of ATP-consuming processes decreased [32]. This mechanism has revealed that AMPK could restore energy homeostasis through an anabolic pathway to consume ATP or catabolic pathways for ATP production [33]. Hence, some clinical/reference drugs such as Metformin and Phenformin could assist peripheral tissues or skeletal muscles to uptake or utilize glucose and even increase insulin sensitivity [5,34].To avoid adverse eﬀects (diarrhea, nausea, ketonemia, etc.)from clinical drugs such as Metformin and Phenformin [5,35], some natural products such as curcumin [36], rutin, quercetin [37], and catechin [38] were applied to battle or ameliorate diabetes. Notably, G. xanthochymus China [39] showed that it was folk medicine used for treating several diseases including diabetes. In previous studies, there were three major compounds identiﬁed: 12b-hydroxy-des-d-garcigerrin, 1,2,5,6-tretrahydroxy-4-(1,1-dimethyl-2-propenyl)-7-(3-methyl-2-butenyl)xanthone,and 1,5,6-trihydroxy-7,8-di(3-methyl-2-butenyl)- $^{\cdot6^{\prime},6^{\prime}}$ -dimethylpyrano  $(2^{\prime}{},3^{\prime}{};\!3,\!4)$  xanthone that were isolated in the extract of G. xanthochymusﬀect on the promotion of glucose uptake in skeletal cells when compared with Metformin [25]. Our docking results showed that  $\alpha$ -tocopherolquinone,  $6\upbeta$ -Hydroxystigmast-4-en-3-one, 1,6-dihydroxy- $^{5,7}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\$ -dimethoxyxanthone,  G. linii (Figure 1) and had a higher binding aﬃnity with AMPK α1 than Metformin. Interestingly, there is a signiﬁcant eﬀect that increases glucose uptake in skeletal cells when compared with Metformin. Alternatively, G. linii alone or in combination with Metformin can be more prospective to alleviate side-eﬀects or elevate applicable time (e.g., cumulative eﬀect) by reducing Metformin dosage for clinical use.

## 1.6. Peroxisome Proliferator-Activated Receptor Gamma (PPARγ)

The peroxisome proliferator-activated receptor (PPAR) is a nuclear receptor superfamily and has three isotypes  $\alpha,\ \delta,$  and  $\gamma$  that can regulate lipid metabolism, inﬂammation, and insulin sensitivity as well as insulin production and secretion for treating diabetes [40–42].PPARγ could mediate lipid mobilization, glucose metabolism, inﬂammatory response, and adipokines production and secretion [41,43]. Henceforth, cumulative studies emerged and showed PPARγ ligands that could promote triglyceride storage in fat that was implicated in insulin resistance and control adipocyte-secreted hormones [41]. In clinical treatments, Rosiglitazone is an agonist of PPARγ that could ameliorate the memory of Alzheimer patients and even increase insulin sensitivity for diabetes [44].In traditional therapy, thiazolidinedione (TZD) was usually used to treat diabetes patients but TZD promotes triglyceride storage that causes adverse eﬀects such as headache, muscle soreness, obesity, edema, etc. [45 G. cambogia -hydroxycitric acid (HCA), which was found to be an active ingredient used to treat obesity and $(-).$  obesity-related diseases, e.g., diabetes, atherosclerosis, etc. [46]. The results (Figure 1) showed that -tocopherolquinone,  $\propto$  $6\upbeta$ -Hydroxystigmast-4-en-3-one, 1,6-Dihydroxy- $^{3,5}\!.$ -dimethoxyxanthen-9-one, and 1,6-Dihydroxy-5-methoxyxanthone stimulated insulin sensitivity, and in virtual screening via the binding aﬃnity of GW9662 (reference drug), which is lower than those of compounds isolated from G. linii.

## 1.7. Dipeptidyl-Peptidase 4 (DPP-4) and Glucagon-Like Peptide 1 (GLP-1)

The dipeptidyl peptidase-4 (DPP-4) could hydrolyze glucagon-like peptide 1 (GLP-1) or gastric inhibitory polypeptide (GIP) and lead to negative eﬀects on the concentration of incretins (GLP-1 and GIP), insulin secretion, and glucose tolerance due to DPP4 gene expression [47]. Consequently, some diabetes patients may take a DPP4 inhibitor such as Sitagliptin to increase insulin secretion for diabetes therapy and ameliorate the therapeutic eﬀect of GLP-1 [48,49]. The GLP-1 was treated with DPP4 inhibitors against diabetes from 2005 to 2007 and still had adverse eﬀects such as rhinopharyngitis and upper respiratory tract infections [50]. Therefore, some cumulative studies indicated that natural compounds, e.g., rutin, curcumin, antroquinonol, quercetin, and 16-hydroxy-cleroda-3, 13-dien-15, 16-olide (HCD), could inhibit DPP4 activity, such as the inhibitory eﬃcacy of curcumin and quercetin, better than Sitagliptin [36,, G. cambogia which contains hydroxycitric acid (HCA), could decrease the serum insulin levels and prolong intestinal tracts to absorb glucose as well as to potentially change incretins (GLP-1, GIP) secretions [10,52]. Taken altogether, the extract of  $G$ . cambogia syndromes, and lead to weight loss. To increase insulin sensitivity, our docking results showed that benzopyrans, triterpenes, stigmastane, and biﬂavonoids were found to act as insulin receptor agonists and promoted glucose uptake in skeletal cells from blood. Hereafter, incretins are degraded by DPP4 and lead to pancreatic  $\upbeta$  cells to decrease secretions of insulin (Figure 1). Of note, the reference drug, Sitagliptin, plays a major role in inhibiting the activation of DPP4. Obviously, our data indicated that -tocopherolquinone and squalene had stronger binding a $\alpha$ ﬃnity with DPP4 as an inhibitor than with Sitagliptin to prevent incretin (GLP-1) degraded by DPP4.



## 1.8. Insulin Receptor Kinase (IRK)

-subunits of insulin receptors receive signal insulin, which triggers tyrosine kinase of $\alpha$   $\upbeta$ -subunits (Insulin receptor kinase, IRK) to form intracellular auto-phosphorylation at Tyr1158, Tyr1162, and Tyr1163 [53]. Once the insulin receptors are activated, they promote PI3K to phosphorylate PIP2; and, further, PIP3 leads the PDK1/2 activation. When AKT was phosphorylated by receiving the signal, the downstream AS160 would prompt glucose transporter 4 (GLUT4) translocation and uptake glucose into the cells [54]. Previously, some natural compounds have been demonstrated such as  Antrodia cinnamomea $(+)\cdot$ 55], rutin (a kind of ﬂavonoid) isolated from Toona sinensis53], and the phenolics isolated from coﬀee silverskins and husks [56] that result in lowered glucose levels. All of these compounds could enhance the activation of IRK to promote the skeletal tissues to absorb glucose and, consequently, ameliorate insulin resistance by reducing blood glucose levels in the diabetic patients. Therefore, in this study, we collated research from the literature by the application of the Garcinia species for various anti-diabetes treatments. Previous literature  G. xanthochymus, G. kola, G. mangostana, G. pedunculata G. prainiana compounds, e.g., biﬂavonoids, xanthone, HCA, and depsidone, which could augment IRK activity and regulate the blood glucose levels for diabetic patients [10,15,16,25,57]. Accordingly, our docking data revealed that only  $\propto$ -tocopherolquinone had a higher binding aﬃnity with IRK than a reference drug (Chaetochromin), suggesting that  $\alpha$ -tocopherolquinone acts as an anti-hyperglycemic compound to heighten IRK activity (Figure 1).

## 2. Conclusions and Future Remarks

In ancient societies, the Garcinia species were used as a daily supply, e.g., building material, food additives, fruit juice, jam, and dye. However, natural compounds that are isolated from the bark, seeds,  G. kola G. pedunculata, G. prainiana, G. mangostana, G. xanthochymus G. cambogia These compounds are applied to treat adipogenesis, inﬂammation, obesity, cancer, cardiovascular diseases, and diabetes. Predominantly, the isolated natural compounds of G. linii in this study are employed to do molecule docking with  $\alpha$ -amylase,  $\propto$ -glucosidase, AMPK, IRK, PPARγ, and DPP4, respectively. Of note, our docking data revealed that the ChemPLP scores for Benzopyrans, Flavonols, Polyphenol, Stigmastane, and Triterpenes isolated from G. linii had a higher AMPK aﬃnity when compared with Metformin; and, alternatively, Garcinia linii alone or in combination with Metformin can have a greater potential to alleviate side-eﬀects or elevate applicable time (e.g., cumulative eﬀect) by reducing Metformin dosage. These results demonstrated that benzopyrans and triterpenes had a stronger binding aﬃnity with anti-diabetic target molecules as a template than reference drugs, e.g., Acarbose with  $\alpha$ -Amylase and  $\alpha$ -Glucosidase, Metformin with AMPK, Sitagliptin with DPP4, Chaetochromin with IRK, and GW9662 with PPARγ. According to this evidence, benzopyrans and triterpenes are suggested to be the active components in G. linii for mediating blood glucose. To further validate the potency of these active components, compounds puriﬁed and subsequently the enzyme activity test, an in vitro cellular function assay and an in vivo animal eﬃcacy experiment need to be conducted to investigate their potential role in anti-diabetes and anti-hyperglycemia in the future.

Author Contributions: Conceptualization, Y.-S.F. and C.-F.W.; Methodology, T.-H.C. and C.-F.W.; Software, T.-H.C. and C.-F.W.; Validation, Y.-S.F. and C.-F.W.; Writing and editing,—original draft preparation, T.-H.C. and M.-J.T.; writing—review M.-J.T. and C.-F.W.; Funding acquisition, C.-F.W.

Funding:This research was funded by Ministry of Science and Technology, Taiwan [grant number 107-2320-B-259-003] for C.F. Weng.

Acknowledgments: We sincerely thanks Max K Leong (Department of Chemistry, NDHU, Hualien, Taiwan) for his help for this work accomplishment.

Conﬂicts of Interest: The authors declare no conﬂict of interest.

## References

1.Hay, S.I.; Abajobir, A.A.; Abate, K.H.; Abbafati, C.; Abbas, K.M.; Abd-Allah, F.; Abdulkader, R.S.; Abdulle, A.M.; Abebo, T.A.; Abera, S.F.; et al. Global, regional, and national disability-adjusted life-years (DALYs) for 333 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories,  Lancet 2017, 390 [CrossRef]

### 2. Global Report on Diabetes

3.Atlas, D.; International Diabetes Federation. IDF Diabetes Atlas Brussels, Belgium, 2015.

4. Vasc. Health Risk Manag. 2008, 4CrossRefPubMed 5.Thomas, I.; Gregg, B. Metformin; a review of its history and future: From lilac to longevity. Pediatr. Diabetes 2017, 18CrossRefPubMed 6.Mendler, M.; Kopf, S.; Groener, J.B.; Riedinger, C.; Fleming, T.H.; Nawroth, P.P.; Okun, J.G. Urine levels of  Acta Diabetol. 2018, 55, 585–592. [CrossRef]

 



7. Drugs 2017, 77CrossRef

8.Abou Daya, K.; Daya, H.A.; Eddine, M.N.; Nahhas, G.; Nuwayri-Salti, N. Eﬀects of rosiglitazone (PPAR  J. Diabetes 2015, 7, 85–94. [CrossRef] 9.Mahajan, U.B.; Chandrayan, G.; Patil, C.R.; Arya, D.S.; Suchal, K.; Agrawal, Y.O.; Ojha, S.; Goyal, S.N. The Protective Eﬀect of Apigenin on Myocardial Injury in Diabetic Rats mediating Activation of the PPAR-gamma  Int. J. Mol. Sci. 2017, 18CrossRef 10.Thazhath, S.S.; Wu, T.; Bound, M.J.; Checklin, H.L.; Standﬁeld, S.; Jones, K.L.; Horowitz, M.; Rayner, C.K. Eﬀects of intraduodenal hydroxycitrate on glucose absorption, incretin release, and glycemia in response to  Nutrition 2016, 32CrossRef 11.Li, Y.; Chen, Y.; Xiao, C.; Chen, D.; Xiao, Y.; Mei, Z. Rapid screening and identiﬁcation of alpha-amylase inhibitors from Garcinia xanthochymus using enzyme-immobilized magnetic nanoparticles coupled with  J. Chromatogr. B Analyt. Technol. Biomed. Life Sci. 2014, 960CrossRef 12.Adaramoye, O.A. Antidiabetic eﬀect of kolaviron, a biﬂavonoid complex isolated from Garcinia kola seeds, in Wistar rats. Afr. Health Sci. 2012, 12CrossRef 13.Karim, N.; Rahman, A.; Chanudom, L.; Thongsom, M.; Tangpong, J. Mangosteen Vinegar Rind from Garcinia mangostana Prevents High-Fat Diet and Streptozotocin-Induced Type II Diabetes Nephropathy  J. Food Sci. 2019, 84CrossRefPubMed 14.Loo, A.E.; Huang, D. Assay-guided fractionation study of alpha-amylase inhibitors from Garcinia mangostana  J. Agric. Food Chem. 2007, 55PubMed 15.Ali, M.Y.; Paul, S.; Tanvir, E.M.; Hossen, M.S.; Rumpa, N.N.; Saha, M.; Bhoumik, N.C.; Islam, M.A.; Hossain, M.S.; Alam, N.; et al. Antihyperglycemic, Antidiabetic, and Antioxidant Eﬀects of Garcinia pedunculata in Rats. Evid.-Based Complement. Altern. Med. 2017, 2017CrossRefPubMed 16.Susanti, D.; Amiroudine, M.Z.; Rezali, M.F.; Taher, M. Friedelin and lanosterol from Garcinia prainiana stimulated glucose uptake and adipocytes diﬀerentiation in 3T3-L1 adipocytes. Nat. Prod. Res. 2013, 27, 417–424. [CrossRef] [PubMed] 17.Inoue, T.; Kainuma, M.; Baba, K.; Oshiro, N.; Kimura, N.; Chan, E.W. Garcinia subelliptica Merr. (Fukugi): A  J. Intercult. Ethnopharmacol. 2017, 6 18.Mohamed, G.A.; Al-Abd, A.M.; El-Halawany, A.M.; Abdallah, H.M.; Ibrahim, S.R.M. New xanthones and cytotoxic constituents from Garcinia mangostana fruit hulls against human hepatocellular, breast, and  J. Ethnopharmacol. 2017, 198CrossRef 19.Semwal, R.B.; Semwal, D.K.; Vermaak, I.; Viljoen, A. A comprehensive scientiﬁc overview of Garcinia  Fitoterapia 2015, 102CrossRef 20.Tousian Shandiz, H.; Razavi, B.M.; Hosseinzadeh, H. Review of Garcinia mangostana and its Xanthones in Metabolic Syndrome and Related Complications. Phytother. Res. 2017, 31CrossRef 21.Alam, F.; Islam, M.A.; Kamal, M.A.; Gan, S.H. Updates on Managing Type 2 Diabetes Mellitus with Natural Products: Towards Antidiabetic Drug Development. Curr. Med. Chem. 2018, 25CrossRef 22.Liu, Q.Y.; Wang, Y.T.; Lin, L.G. New insights into the anti-obesity activity of xanthones from Garcinia  Food Funct. 2015, 6CrossRef 23.Nguyen, C.N.; Trinh, B.T.D.; Tran, T.B.; Nguyen, L.T.; Jager, A.K.; Nguyen, L.D. Anti-diabetic xanthones from the bark of Garcinia xanthochymus. Bioorg. Med. Chem. Lett. 2017, 27CrossRefPubMed 24.Widowati, W.; Laksmitawati, D.R.; Wargasetia, T.L.; Aﬁfah, E.; Amalia, A.; Arinta, Y.; Rizal, R.; Suciati, T. Mangosteen peel extract (Garcinia mangostana L. ) as protective agent in glucose-induced mesangial cell as  Iran. J. Basic Med. Sci. 2018, 21PubMed 25.Li, Y.; Zhao, P.; Chen, Y.; Fu, Y.; Shi, K.; Liu, L.; Liu, H.; Xiong, M.; Liu, Q.H.; Yang, G.; et al. Depsidone and xanthones from Garcinia xanthochymus with hypoglycemic activity and the mechanism of promoting glucose uptake in L6 myotubes. Bioorg. Med. Chem. 2017, 25CrossRefPubMed 26.Maia-Landim, A.; Ramirez, J.M.; Lancho, C.; Poblador, M.S.; Lancho, J.L. Long-term eﬀects of Garcinia cambogia/Glucomannan on weight loss in people with obesity, PLIN4, FTO and Trp64Arg polymorphisms. BMC Complement. Altern. Med. 2018, 18CrossRef 27.Liu, S.; Li, D.; Huang, B.; Chen, Y.; Lu, X.; Wang, Y. Inhibition of pancreatic lipase, alpha-glucosidase, alpha-amylase, and hypolipidemic eﬀects of the total ﬂavonoids from Nelumbo nucifera leaves. J. Ethnopharmacol. 2013, 149CrossRef 28.Chen, J.J.; Chen, I.S.; Duh, C.Y. Cytotoxic xanthones and biphenyls from the root of Garcinia linii. Planta Med. 2004, 70CrossRef 29.Chen, J.J.; Peng, C.F.; Huang, H.Y.; Chen, I.S. Benzopyrans, biphenyls and xanthones from the root of Garcinia linii and their activity against Mycobacterium tuberculosis. Planta Med. 2006, 72CrossRef 30.Pantidos, N.; Boath, A.; Lund, V.; Conner, S.; McDougall, G.J. Phenolic-rich extracts from the edible seaweed, ascophyllum nodosum, inhibit  $\alpha$ -amylase and  $\alpha$ -glucosidase: Potential anti-hyperglycemic eﬀects. J. Funct. Foods 2014, 10CrossRef 31.Hardie, D.G.; Schaﬀer, B.E.; Brunet, A. AMPK: An Energy-Sensing Pathway with Multiple Inputs and Outputs. Trends Cell Biol. 2016, 26CrossRef 32. Biochem. Pharmacol. 2018, 153CrossRef 33.Scott, J.W.; Ling, N.; Issa, S.M.; Dite, T.A.; O’Brien, M.T.; Chen, Z.P.; Galic, S.; Langendorf, C.G.; Steinberg, G.R.; Kemp, B.E.; et al. Small molecule drug A-769662 and AMP synergistically activate naive AMPK independent of upstream kinase signaling. Chem. Biol. 2014, 21CrossRefPubMed 34.Oh-Hashi, K.; Irie, N.; Sakai, T.; Okuda, K.; Nagasawa, H.; Hirata, Y.; Kiuchi, K. Elucidation of a novel phenformin derivative on glucose-deprived stress responses in HT-29 cells. Mol. Cell. Biochem. 2016, 419, 29–40. [CrossRef] [PubMed] 35.Dujic, T.; Causevic, A.; Bego, T.; Malenica, M.; Velija-Asimi, Z.; Pearson, E.R.; Semiz, S. Organic cation transporter 1 variants and gastrointestinal side eﬀects of metformin in patients with Type 2 diabetes. Diabet. Med. 2016, 33CrossRefPubMed 36.Seo, K.I.; Choi, M.S.; Jung, U.J.; Kim, H.J.; Yeo, J.; Jeon, S.M.; Lee, M.K. Eﬀect of curcumin supplementation on blood glucose, plasma insulin, and glucose homeostasis related enzyme activities in diabetic db/db mice. Mol. Nutr. Food Res. 2008, 52CrossRefPubMed 37.Nazeri, S.; Farhangi, M.; Modarres, S. The eﬀect of diﬀerent dietary inclusion levels of rutin (a ﬂavonoid) on some liver enzyme activities and oxidative stress indices in rainbow trout, Oncorhynchus mykiss (Walbaum) exposed to Oxytetracycline. Aquac. Res. 2017, 48CrossRef 38.Meena, K.P.; Vijayakumar, M.R.; Dwibedy, P.S. Catechin-loaded Eudragit microparticles for the management of diabetes: Formulation, characterization and in vivo evaluation of antidiabetic eﬃcacy. J. Microencapsul. 2017, 34CrossRefPubMed 39.Che Hassan, N.K.N.; Taher, M.; Susanti, D. Phytochemical constituents and pharmacological properties of Garcinia xanthochymus—A review. Biomed. Pharmacother. 2018, 106CrossRef 40.Wang, Q.; Imam, M.U.; Yida, Z.; Wang, F. Peroxisome Proliferator-Activated Receptor Gamma (PPARgamma) as a Target for Concurrent Management of Diabetes and Obesity-Related Cancer. Curr. Pharm. Des. 2017, 23, 3677–3688. [CrossRef] 41.Janani, C.; Kumari, B.D.R. PPAR gamma gene—A review. Diabetes Metab. Syndr. 2015, 9CrossRef 42.Polvani, S.; Tarocchi, M.; Tempesti, S.; Bencini, L.; Galli, A. Peroxisome proliferator activated receptors at the  World J. Gastroenterol. 2016, 22CrossRef 43.Siersbaek, R.; Nielsen, R.; Mandrup, S. PPARgamma in adipocyte diﬀerentiation and metabolism–novel insights from genome-wide studies. FEBS Lett. 2010, 584CrossRefPubMed 44.Sebastiao, I.; Candeias, E.; Santos, M.S.; de Oliveira, C.R.; Moreira, P.I.; Duarte, A.I. Insulin as a Bridge between Type 2 Diabetes and Alzheimer Disease - How Anti-Diabetics Could be a Solution for Dementia. Front. Endocrinol. 2014, 5CrossRefPubMed 45.Elaidy, S.M.; Hussain, M.A.; El-Kherbetawy, M.K. Time-dependent therapeutic roles of nitazoxanide on high-fat diet/streptozotocin-induced diabetes in rats: Eﬀects on hepatic peroxisome proliferator-activated  Can. J. Physiol. Pharmacol. 2018, 96CrossRefPubMed 46.Sharma, K.; Kang, S.; Gong, D.; Oh, S.H.; Park, E.Y.; Oak, M.H.; Yi, E. Combination of Garcinia cambogia Extract and Pear Pomace Extract Additively Suppresses Adipogenesis and Enhances Lipolysis in 3T3-L1  Pharmacogn. Mag. 2018, 14 47.Bohm, A.; Wagner, R.; Machicao, F.; Holst, J.J.; Gallwitz, B.; Stefan, N.; Fritsche, A.; Haring, H.U.; Staiger, H. DPP4 gene variation aﬀects GLP-1 secretion, insulin secretion, and glucose tolerance in humans with high body adiposity. PLoS ONE 2017, 12CrossRef 48.Nahon, K.J.; Doornink, F.; Straat, M.E.; Botani, K.; Martinez-Tellez, B.; Abreu-Vieira, G.; van Klinken, J.B.; Voortman, G.J.; Friesema, E.C.H.; Ruiz, J.R.; et al. Eﬀect of sitagliptin on energy metabolism and brown adipose  Diabetologia 2018, 61CrossRef 49.Araujo, F.; Shrestha, N.; Gomes, M.J.; Herranz-Blanco, B.; Liu, D.; Hirvonen, J.J.; Granja, P.L.; Santos, H.A.; Sarmento, B. In vivo dual-delivery of glucagon like peptide-1 (GLP-1) and dipeptidyl peptidase-4 (DPP4)  Nanoscale 2016, 8 [CrossRef] 50.Andel, M.; Skrha, P.; Kraml, P.; Potockova, J.; Hoﬀmanova, I.; Silhova, E.; Fontana, J.; Richterova, A.; Gadiredi, M.; Busek, P.; et al. Annual monitoring of side eﬀects of administering sitagliptin in patients with type 2 diabetes mellitus. Vnitr. Lék. 2016, 62 51.Huang, P.K.; Lin, S.X.; Tsai, M.J.; Leong, M.K.; Lin, S.R.; Kankala, R.K.; Lee, C.H.; Weng, C.F. Encapsulation of 16-Hydroxycleroda-3,13-Dine-16,15-Olide in Mesoporous Silica Nanoparticles as a Natural Dipeptidyl Peptidase-4 Inhibitor Potentiated Hypoglycemia in Diabetic Mice. Nanomaterials 2017, 7CrossRef 52.Hayamizu, K.; Hirakawa, H.; Oikawa, D.; Nakanishi, T.; Takagi, T.; Tachibana, T.; Furuse, M. Eﬀect of Garcinia  Fitoterapia 2003, 74CrossRef 53.Hsu, C.-Y.; Shih, H.-Y.; Chia, Y.-C.; Lee, C.-H.; Ashida, H.; Lai, Y.-K.; Weng, C.-F. Rutin potentiates insulin receptor kinase to enhance insulin-dependent glucose transporter 4 translocation. Mol. Nutr. Food Res. 2014, 58CrossRefPubMed 54. ´Swiderska, E.; Strycharz, J.; Wróblewski, A.; Szemraj, J.; Drzewoski, J.; ´Sliwi´nska, A. Role of PI3K/AKT Pathway in Insulin-Mediated Glucose Uptake. Glucose Transport [Working Title] online: https://www.intechopen.com/online-ﬁrst/role-of-pi3k-akt-pathway-in-insulin-mediated-glucose- uptake (accessed on 23 October 2019). [CrossRef] 55.Hsu, C.Y.; Sulake, R.S.; Huang, P.K.; Shih, H.Y.; Sie, H.W.; Lai, Y.K.; Chen, C.; Weng, C.F. Synthetic -antroquinonol exhibits dual actions against insulin resistance by triggering AMP kinase and inhibiting $(+)$   Br. J. Pharmacol. 2015, 172CrossRefPubMed 56.Rebollo-Hernanz, M.; Zhang, Q.; Aguilera, Y.; Martin-Cabrejas, M.A.; de Mejia, E.G. Phenolic compounds from coﬀee by-products modulate adipogenesis-related inﬂammation, mitochondrial dysfunction, and insulin resistance in adipocytes, via insulin/PI3K/ Food Chem. Toxicol. 2019, 132, 110672. [CrossRef] [PubMed] 57.Taher, M.; Zakaria, T.M.F.S.T.; Susanti, D.; Zakaria, Z.A. Hypoglycaemic activity of ethanolic extract of  BMC Complement. Altern. Med. 2016, 16 58.Asghar, M.; Monjok, E.; Kouamou, G.; Ohia, S.E.; Bagchi, D.; Lokhandwala, M.F. Super CitriMax (HCA-SX) attenuates increases in oxidative stress, inﬂammation, insulin resistance, and body weight in developing obese Zucker rats. Mol. Cell. Biochem. 2007, 304CrossRef 59.Talpur, N.; Echard, B.W.; Yasmin, T.; Bagchi, D.; Preuss, H.G. Eﬀects of niacin-bound chromium, Maitake mushroom fraction SX and (-)-hydroxycitric acid on the metabolic syndrome in aged diabetic Zucker fatty rats. Mol. Cell. Biochem. 2003, 252CrossRef  by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access $\copyright2019$  article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

                                                   

                                                  

                                                 

                                                

                                               

                                              

                                             

                                            

                                           

                                          

                                         

                                        

                                       

                                      

                                     

                                    

                                   

                                  

                                 

                                

                               

                              

                             

                            

                           

                          

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 


# Review The Exploration of Natural Compounds for Anti-Diabetes from Distinctive Species Garcinia linii with Comprehensive Review of the Garcinia Family

Ting-Hsu Chen , May-Jywan Tsai , Yaw-Syan  $\mathbf{Fu}^{3,4}$  and Ching-Feng Weng \*

1 Department of Life Science and Institute of Biotechnology, National Dong Hwa University, Hualien 97401, Taiwan; 410613031@gms.ndhu.edu.tw 2 Neural Regeneration Laboratory, Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, Taipei 11221, Taiwan; mjtsai2@vghtpe.gov.tw 3 Department of Biomedical Science and Environmental Biology, Kaohsiung Medical University, Kaohsiung city 80708, Taiwan; m805004@kmu.edu.tw 4 Institute of Respiratory Disease, Department of Basic Medical Science, Xiamen Medical College, Xiamen 361023, China \*Correspondence: cfweng-cfweng@hotmail.com; Tel.: +886-3-890-3609; Fax: +886-3-890-0163 Received: 17 September 2019; Accepted: 21 October 2019; Published: 23 October 2019



 

Abstract: Approximately 400 Garcinia species are distributed around the world. Previous studies have reported the extracts from bark, seed, fruits, peels, leaves, and stems of Garcinia mangostana, G. xanthochymus G. cambogia cardiovascular diseases, and diabetes. Moreover, the hypoglycemic eﬀects and underlined actions of diﬀerent species such as G. kola G. pedunculata G. prainiana the anti-hyperglycemia of  $G$ . linii remains to be veriﬁed in this aspect. In this article, the published literature was collected and reviewed based on the medicinal characteristics of the species Garcinia, particularly in diabetic care to deliberate the known constituents from Garcinia and further focus on and isolate new compounds of G. linii (Taiwan distinctive species) on various hypoglycemic targets including  $\alpha$ -amylase,  $\alpha$ -glucosidase,  $5^{\prime}.$ -adenosine monophosphate-activated protein kinase (AMPK), insulin receptor kinase, peroxisome proliferator-activated receptor gamma (PPARγ), and dipeptidyl peptidase-4 (DPP-4) via the molecular docking approach with Gold program to explore the potential candidates for anti-diabetic treatments. Accordingly, benzopyrans and triterpenes are postulated to be the active components in G. linii for mediating blood glucose. To further validate the potency of those active components, in vitro enzymatic and cellular function assays with in vivo animal eﬃcacy experiments need to be performed in the near future.

Keywords: Garcinia linii; hypoglycemia; benzopyran; triterpene; bioﬂavonoid; phenolic; in silico

## 1. Introduction

## 1.1. Impact of Diabetes

Diabetes mellitus (DM) is a global health issue due to its high risk factors, e.g., obesity, physical inactivity, ageing, bad eating habits, genetic predisposition, hypertension, and hyperlipidemia [1]. It is worth noting that DM is a metabolic disease and more than 400 million people suﬀered from diabetes in 2014 [2]. Interestingly, the adult diabetic population of 2014 has risen from  $4.7\%$  to  $8.5\%$  worldwide in contrast to the population in 1980 [2]; the morbidity and mortality were  $4.95\%$  and  $4.00\%,$  respectively [3]. In the future, the diabetic population will increase to 642 million by around 2045 and this population will continue to grow [3].

## 1.2. Therapy Agent of Diabetes

Nowadays, many clinical medicines such as  $\alpha\cdot$ -glucosidase and  $\alpha$ -amylase inhibitors such as Acarbose are applied to delay the metabolism of carbohydrates and control post-meal blood glucose for diabetes patients [4]. Metformin and 5-Amino-4-Imidazolecarboxamide Riboside (AICAR) are activators that increase the glucose absorption of skeletal muscles and inhibit gluconeogenesis of the liver [5,6]. The major anti-diabetes functions of Sitagliptin increase concentrations of incretin. Incretin can further potentiate the pancreas to produce insulin and inhibit the production of glucagon to decrease blood glucose levels [7]. Rosiglitazone can reduce the resistance of insulin absorption in liver cells, skeletal muscle cells, and fatty tissues [8]. Additionally, GW-9662 is an agonist that promotes peroxisome proliferator-activated receptors gamma (PPARγ) expression, which could stimulate fat metabolism, and trigger insulin pathways to regulate blood glucose and has anti-inﬂammatory properties [9].

## 1.3. Distribution of Garcinia Plants and Recent Discovery of Anti-Diabetic Agents with Garcinia Plants

Approximately 400 Garcinia species are distributed around the world including Bangladesh, China, India, Indonesia, Taiwan, Thailand, tropical Asia, Southern Africa, and Western Polynesia. Usually, the Garcinia plants are shrubs or trees. Most of their edible fruits are used in agricultural societies or the fruits and seeds are used to produce oils and dyes, and to treat various diseases, e.g., abdominal pain, food allergies, arthritis, diarrhea, dysentery, and wound infections as past research has shown [10–16 Garcinia subelliptica used as building materials to prevent destruction by typhoons in ancient Japan, which is in contrast to  G. subelliptica17,]. Moreover,   $G$ . mangostana, G. xanthochymus G. cambogia for calories or nutrition and use the extracts of G. xanthochymus G. cambogia increase the sour ﬂavor in India. Interestingly, the extract of G. cambogia preserving food freshness [18,19].

The accumulated literature showed that metabolic syndromes gradually became public health problems such as obesity, hyperglycemia, hyperlipidemia, and hypertension and lead to cardiovascular diseases (i.e., atherosclerosis, stroke, or peripheral artery disease) or diabetes [20]. Hereafter, the medicinal plants or herbs have important characteristics that resist the threat of these diseases because the adverse side eﬀects of natural compounds isolated from medicinal plants are reduced and often less severe than those from clinical drugs [21]. Interestingly, previous studies have reported that the   $G$ . mangostana, G. xanthochymus G. cambogia diabetes. Furthermore, these extracts could also trigger the myotubes and skeletal cells to absorb glucose and to balance blood glucose levels [20,22–26]. Remarkably, numerous studies have indicated that the Garcinia species, e.g., G. cambogia10 G. xanthochymus11 G. kola12 G. mangostana13,], G. pedunculata15 G. prainiana16] contain plenty of biﬂavonoids and phenolic compounds. These compounds have been found to inhibit the enzymatic activity of  $\alpha$ -amylase and  $\alpha$ -glucosidase for propagating an anti-diabetic eﬀect [27]. Concurrently, the variability of biological actions includes anti-diabetic agents that are dependent on the constituents of plants that grow locality, diﬀerently used parts (root, leaves, ﬂowers), or seasonal harvests that exhibit various compositions or ratios for each component, metabolite, or derivative including diﬀerent delivery systems such as powers, pills, or teas, among others. Generally, batch to batch control or a standard operating system via index chemicals or ﬁngerprints is crucial for the evaluation of the eﬃcacy of crude extracts, the yields in isolation, and the puriﬁcation of active ingredients. In this article, the published articles were collected and reviewed. We have addressed the isolated compounds for all Garcinia species by molecular structure and brieﬂy described their targeted biomolecules for anti-diabetic function according to chemical category via PubChem such as benzophenone (one compound isolated from G. mangostana G. mangostana  G. kola G. mangostana compounds isolated from G. xanthochymus G. hanburyi  G. mangostana species for the care of DM (Table 1) are listed to deliberate on the potential constituents of certain Garcinia species. Interestingly, G. linii is one endemic evergreen tree only distributed in outlying islands—Langyu land and Green island of Taiwan. To the best of our knowledge, active constituents including 15 xanthones, 6 biphenyls, 2 benzopyran, and 13 known compounds isolated from the root of  $G$ . linii have been reported with anti-tubercular activity and cytotoxicity [28,]. However, the medicinal values of G. linii extract as an anti-diabetic agent remain to be explored. Moreover, this review article simultaneously oﬀers insights into dissecting the molecular mechanism of isolated compounds such as the three new xanthones (linixanthones A–C), ﬁve new biphenyls (garcibiphenyls A–E), and two new benzopyran (garcibenzopyran and (S)-3-hydroxygarcibenzopyran) of the G. linii root from Taiwan [28,O-Methylmacluraxanthone;  $^{1,5}\!.$ -Dihydroxyxanthone; 1,6-Dihydroxy-3,5,7-trimethoxyxanthone; 1,6-Dihydroxy-5-methoxyxanthone; 1,6-dihydroxy- $^{5,7}\!.$ -dimethoxyxanthone; 1,6-Dihydroxy-7-methoxyxanthone; 1,7-Dihydroxyxanthone; 1,7-Dihydroxy-3-methoxyxanthone;5-Hydroxy-1-methoxyxanthone)fromtheGarcinia family on various hypoglycemic targets including  $\alpha$ -amylase,-glucosidase, $\propto$ -adenosine $5^{\prime}.$  monophosphate-activated protein kinase (AMPK), insulin receptor kinase (IRK), PPARγ, and dipeptidyl peptidase-4 (DPP4) via the molecular docking approach with Gold program (Cambridge Crystallographic Data Centre, Cambridge, UK).



## 1.4. α-Amylase and  $\alpha$ -Glucosidase

To regulate the postprandial blood glucose level, diabetic patients took carbohydrate hydrolase inhibitors such as  $\alpha$ -glucosidase and  $\alpha$ -amylase to avoid hyperglycemia.  $\alpha$ -amylase and  $\alpha$ -glucosidase are the key enzymes to hydrolyze carbohydrates and help glucose ingestion [30]. Therefore, diabetic patients have to control their blood glucose by using clinical drugs such as  $P r e c o s e^{\circled{R}}$  (Acarbose) and  $G l y s e t^{\widehat{\mathtt{Q}}}$  (Miglitol) [4] or other anti-diabetic natural compounds [27] to prolong hydrolysis of carbohydrates against hyperglycemia. In cumulative studies, a few crude extracts from the Garcinia species, e.g., G. cambogia G. xanthochymus G. kola G. mangostana10–14], containing biﬂavonoids, polyphenols, and xanthones, also inhibit the enzyme activity of  $\alpha$ -amylase and  $\propto$ -glucosidase. Therefore, the extracts are able to help diabetic patients to control their blood glucose levels by the inhibition of carbohydrate hydrolysis. Our docking results (Figure 1) showed that benzopyrans and triterpenes had a higher binding aﬃnity with  $\alpha$ -amylase and  $\alpha$ -glucosidase than with biﬂavonoid and phenolic compounds. Additionally,  $\alpha$ -tocopherolquinone (a kind of benzopyrans) and squalene (a kind of triterpenes) had a high binding aﬃnity with  $\alpha$ -amylase and  $\alpha$ -glucosidase to prolong carbohydrate hydrolyzation, reduce the absorption of glucose and mediate the blood glucose level.

## 1.5.  $5^{\prime}$ -Adenosine Monophosphate-Activated Protein Kinase (AMPK)

The  $5^{\prime}$ -adenosine monophosphate-activated protein kinase (AMPK) is composed of  $\alpha$ subunits, regulatory  $\upbeta$  subunits, and r subunits and is a sensor of cellular energy level. Cellular energy levels were changed by the ratio of AMP:ATP and ADP:ATP that inﬂuenced cellular growth and survival [31]. Previous research indicated that AMPK was activated and the rate of ATP-generating processes would increase while the rate of ATP-consuming processes decreased [32]. This mechanism has revealed that AMPK could restore energy homeostasis through an anabolic pathway to consume ATP or catabolic pathways for ATP production [33]. Hence, some clinical/reference drugs such as Metformin and Phenformin could assist peripheral tissues or skeletal muscles to uptake or utilize glucose and even increase insulin sensitivity [5,34].To avoid adverse eﬀects (diarrhea, nausea, ketonemia, etc.)from clinical drugs such as Metformin and Phenformin [5,35], some natural products such as curcumin [36], rutin, quercetin [37], and catechin [38] were applied to battle or ameliorate diabetes. Notably, G. xanthochymus China [39] showed that it was folk medicine used for treating several diseases including diabetes. In previous studies, there were three major compounds identiﬁed: 12b-hydroxy-des-d-garcigerrin, 1,2,5,6-tretrahydroxy-4-(1,1-dimethyl-2-propenyl)-7-(3-methyl-2-butenyl)xanthone,and 1,5,6-trihydroxy-7,8-di(3-methyl-2-butenyl)- $^{\cdot6^{\prime},6^{\prime}}$ -dimethylpyrano  $(2^{\prime}{},3^{\prime}{};\!3,\!4)$  xanthone that were isolated in the extract of G. xanthochymusﬀect on the promotion of glucose uptake in skeletal cells when compared with Metformin [25]. Our docking results showed that  $\alpha$ -tocopherolquinone,  $6\upbeta$ -Hydroxystigmast-4-en-3-one, 1,6-dihydroxy- $^{5,7}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\$ -dimethoxyxanthone,  G. linii (Figure 1) and had a higher binding aﬃnity with AMPK α1 than Metformin. Interestingly, there is a signiﬁcant eﬀect that increases glucose uptake in skeletal cells when compared with Metformin. Alternatively, G. linii alone or in combination with Metformin can be more prospective to alleviate side-eﬀects or elevate applicable time (e.g., cumulative eﬀect) by reducing Metformin dosage for clinical use.

## 1.6. Peroxisome Proliferator-Activated Receptor Gamma (PPARγ)

The peroxisome proliferator-activated receptor (PPAR) is a nuclear receptor superfamily and has three isotypes  $\alpha,\ \delta,$  and  $\gamma$  that can regulate lipid metabolism, inﬂammation, and insulin sensitivity as well as insulin production and secretion for treating diabetes [40–42].PPARγ could mediate lipid mobilization, glucose metabolism, inﬂammatory response, and adipokines production and secretion [41,43]. Henceforth, cumulative studies emerged and showed PPARγ ligands that could promote triglyceride storage in fat that was implicated in insulin resistance and control adipocyte-secreted hormones [41]. In clinical treatments, Rosiglitazone is an agonist of PPARγ that could ameliorate the memory of Alzheimer patients and even increase insulin sensitivity for diabetes [44].In traditional therapy, thiazolidinedione (TZD) was usually used to treat diabetes patients but TZD promotes triglyceride storage that causes adverse eﬀects such as headache, muscle soreness, obesity, edema, etc. [45 G. cambogia -hydroxycitric acid (HCA), which was found to be an active ingredient used to treat obesity and $(-).$  obesity-related diseases, e.g., diabetes, atherosclerosis, etc. [46]. The results (Figure 1) showed that -tocopherolquinone,  $\propto$  $6\upbeta$ -Hydroxystigmast-4-en-3-one, 1,6-Dihydroxy- $^{3,5}\!.$ -dimethoxyxanthen-9-one, and 1,6-Dihydroxy-5-methoxyxanthone stimulated insulin sensitivity, and in virtual screening via the binding aﬃnity of GW9662 (reference drug), which is lower than those of compounds isolated from G. linii.

## 1.7. Dipeptidyl-Peptidase 4 (DPP-4) and Glucagon-Like Peptide 1 (GLP-1)

The dipeptidyl peptidase-4 (DPP-4) could hydrolyze glucagon-like peptide 1 (GLP-1) or gastric inhibitory polypeptide (GIP) and lead to negative eﬀects on the concentration of incretins (GLP-1 and GIP), insulin secretion, and glucose tolerance due to DPP4 gene expression [47]. Consequently, some diabetes patients may take a DPP4 inhibitor such as Sitagliptin to increase insulin secretion for diabetes therapy and ameliorate the therapeutic eﬀect of GLP-1 [48,49]. The GLP-1 was treated with DPP4 inhibitors against diabetes from 2005 to 2007 and still had adverse eﬀects such as rhinopharyngitis and upper respiratory tract infections [50]. Therefore, some cumulative studies indicated that natural compounds, e.g., rutin, curcumin, antroquinonol, quercetin, and 16-hydroxy-cleroda-3, 13-dien-15, 16-olide (HCD), could inhibit DPP4 activity, such as the inhibitory eﬃcacy of curcumin and quercetin, better than Sitagliptin [36,, G. cambogia which contains hydroxycitric acid (HCA), could decrease the serum insulin levels and prolong intestinal tracts to absorb glucose as well as to potentially change incretins (GLP-1, GIP) secretions [10,52]. Taken altogether, the extract of  $G$ . cambogia syndromes, and lead to weight loss. To increase insulin sensitivity, our docking results showed that benzopyrans, triterpenes, stigmastane, and biﬂavonoids were found to act as insulin receptor agonists and promoted glucose uptake in skeletal cells from blood. Hereafter, incretins are degraded by DPP4 and lead to pancreatic  $\upbeta$  cells to decrease secretions of insulin (Figure 1). Of note, the reference drug, Sitagliptin, plays a major role in inhibiting the activation of DPP4. Obviously, our data indicated that -tocopherolquinone and squalene had stronger binding a $\alpha$ ﬃnity with DPP4 as an inhibitor than with Sitagliptin to prevent incretin (GLP-1) degraded by DPP4.



## 1.8. Insulin Receptor Kinase (IRK)

-subunits of insulin receptors receive signal insulin, which triggers tyrosine kinase of $\alpha$   $\upbeta$ -subunits (Insulin receptor kinase, IRK) to form intracellular auto-phosphorylation at Tyr1158, Tyr1162, and Tyr1163 [53]. Once the insulin receptors are activated, they promote PI3K to phosphorylate PIP2; and, further, PIP3 leads the PDK1/2 activation. When AKT was phosphorylated by receiving the signal, the downstream AS160 would prompt glucose transporter 4 (GLUT4) translocation and uptake glucose into the cells [54]. Previously, some natural compounds have been demonstrated such as  Antrodia cinnamomea $(+)\cdot$ 55], rutin (a kind of ﬂavonoid) isolated from Toona sinensis53], and the phenolics isolated from coﬀee silverskins and husks [56] that result in lowered glucose levels. All of these compounds could enhance the activation of IRK to promote the skeletal tissues to absorb glucose and, consequently, ameliorate insulin resistance by reducing blood glucose levels in the diabetic patients. Therefore, in this study, we collated research from the literature by the application of the Garcinia species for various anti-diabetes treatments. Previous literature  G. xanthochymus, G. kola, G. mangostana, G. pedunculata G. prainiana compounds, e.g., biﬂavonoids, xanthone, HCA, and depsidone, which could augment IRK activity and regulate the blood glucose levels for diabetic patients [10,15,16,25,57]. Accordingly, our docking data revealed that only  $\propto$ -tocopherolquinone had a higher binding aﬃnity with IRK than a reference drug (Chaetochromin), suggesting that  $\alpha$ -tocopherolquinone acts as an anti-hyperglycemic compound to heighten IRK activity (Figure 1).

## 2. Conclusions and Future Remarks

In ancient societies, the Garcinia species were used as a daily supply, e.g., building material, food additives, fruit juice, jam, and dye. However, natural compounds that are isolated from the bark, seeds,  G. kola G. pedunculata, G. prainiana, G. mangostana, G. xanthochymus G. cambogia These compounds are applied to treat adipogenesis, inﬂammation, obesity, cancer, cardiovascular diseases, and diabetes. Predominantly, the isolated natural compounds of G. linii in this study are employed to do molecule docking with  $\alpha$ -amylase,  $\propto$ -glucosidase, AMPK, IRK, PPARγ, and DPP4, respectively. Of note, our docking data revealed that the ChemPLP scores for Benzopyrans, Flavonols, Polyphenol, Stigmastane, and Triterpenes isolated from G. linii had a higher AMPK aﬃnity when compared with Metformin; and, alternatively, Garcinia linii alone or in combination with Metformin can have a greater potential to alleviate side-eﬀects or elevate applicable time (e.g., cumulative eﬀect) by reducing Metformin dosage. These results demonstrated that benzopyrans and triterpenes had a stronger binding aﬃnity with anti-diabetic target molecules as a template than reference drugs, e.g., Acarbose with  $\alpha$ -Amylase and  $\alpha$ -Glucosidase, Metformin with AMPK, Sitagliptin with DPP4, Chaetochromin with IRK, and GW9662 with PPARγ. According to this evidence, benzopyrans and triterpenes are suggested to be the active components in G. linii for mediating blood glucose. To further validate the potency of these active components, compounds puriﬁed and subsequently the enzyme activity test, an in vitro cellular function assay and an in vivo animal eﬃcacy experiment need to be conducted to investigate their potential role in anti-diabetes and anti-hyperglycemia in the future.

Author Contributions: Conceptualization, Y.-S.F. and C.-F.W.; Methodology, T.-H.C. and C.-F.W.; Software, T.-H.C. and C.-F.W.; Validation, Y.-S.F. and C.-F.W.; Writing and editing,—original draft preparation, T.-H.C. and M.-J.T.; writing—review M.-J.T. and C.-F.W.; Funding acquisition, C.-F.W.

Funding:This research was funded by Ministry of Science and Technology, Taiwan [grant number 107-2320-B-259-003] for C.F. Weng.

Acknowledgments: We sincerely thanks Max K Leong (Department of Chemistry, NDHU, Hualien, Taiwan) for his help for this work accomplishment.

Conﬂicts of Interest: The authors declare no conﬂict of interest.

## References

1.Hay, S.I.; Abajobir, A.A.; Abate, K.H.; Abbafati, C.; Abbas, K.M.; Abd-Allah, F.; Abdulkader, R.S.; Abdulle, A.M.; Abebo, T.A.; Abera, S.F.; et al. Global, regional, and national disability-adjusted life-years (DALYs) for 333 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories,  Lancet 2017, 390 [CrossRef]

### 2. Global Report on Diabetes

3.Atlas, D.; International Diabetes Federation. IDF Diabetes Atlas Brussels, Belgium, 2015.

4. Vasc. Health Risk Manag. 2008, 4CrossRefPubMed 5.Thomas, I.; Gregg, B. Metformin; a review of its history and future: From lilac to longevity. Pediatr. Diabetes 2017, 18CrossRefPubMed 6.Mendler, M.; Kopf, S.; Groener, J.B.; Riedinger, C.; Fleming, T.H.; Nawroth, P.P.; Okun, J.G. Urine levels of  Acta Diabetol. 2018, 55, 585–592. [CrossRef]

 



7. Drugs 2017, 77CrossRef

8.Abou Daya, K.; Daya, H.A.; Eddine, M.N.; Nahhas, G.; Nuwayri-Salti, N. Eﬀects of rosiglitazone (PPAR  J. Diabetes 2015, 7, 85–94. [CrossRef] 9.Mahajan, U.B.; Chandrayan, G.; Patil, C.R.; Arya, D.S.; Suchal, K.; Agrawal, Y.O.; Ojha, S.; Goyal, S.N. The Protective Eﬀect of Apigenin on Myocardial Injury in Diabetic Rats mediating Activation of the PPAR-gamma  Int. J. Mol. Sci. 2017, 18CrossRef 10.Thazhath, S.S.; Wu, T.; Bound, M.J.; Checklin, H.L.; Standﬁeld, S.; Jones, K.L.; Horowitz, M.; Rayner, C.K. Eﬀects of intraduodenal hydroxycitrate on glucose absorption, incretin release, and glycemia in response to  Nutrition 2016, 32CrossRef 11.Li, Y.; Chen, Y.; Xiao, C.; Chen, D.; Xiao, Y.; Mei, Z. Rapid screening and identiﬁcation of alpha-amylase inhibitors from Garcinia xanthochymus using enzyme-immobilized magnetic nanoparticles coupled with  J. Chromatogr. B Analyt. Technol. Biomed. Life Sci. 2014, 960CrossRef 12.Adaramoye, O.A. Antidiabetic eﬀect of kolaviron, a biﬂavonoid complex isolated from Garcinia kola seeds, in Wistar rats. Afr. Health Sci. 2012, 12CrossRef 13.Karim, N.; Rahman, A.; Chanudom, L.; Thongsom, M.; Tangpong, J. Mangosteen Vinegar Rind from Garcinia mangostana Prevents High-Fat Diet and Streptozotocin-Induced Type II Diabetes Nephropathy  J. Food Sci. 2019, 84CrossRefPubMed 14.Loo, A.E.; Huang, D. Assay-guided fractionation study of alpha-amylase inhibitors from Garcinia mangostana  J. Agric. Food Chem. 2007, 55PubMed 15.Ali, M.Y.; Paul, S.; Tanvir, E.M.; Hossen, M.S.; Rumpa, N.N.; Saha, M.; Bhoumik, N.C.; Islam, M.A.; Hossain, M.S.; Alam, N.; et al. Antihyperglycemic, Antidiabetic, and Antioxidant Eﬀects of Garcinia pedunculata in Rats. Evid.-Based Complement. Altern. Med. 2017, 2017CrossRefPubMed 16.Susanti, D.; Amiroudine, M.Z.; Rezali, M.F.; Taher, M. Friedelin and lanosterol from Garcinia prainiana stimulated glucose uptake and adipocytes diﬀerentiation in 3T3-L1 adipocytes. Nat. Prod. Res. 2013, 27, 417–424. [CrossRef] [PubMed] 17.Inoue, T.; Kainuma, M.; Baba, K.; Oshiro, N.; Kimura, N.; Chan, E.W. Garcinia subelliptica Merr. (Fukugi): A  J. Intercult. Ethnopharmacol. 2017, 6 18.Mohamed, G.A.; Al-Abd, A.M.; El-Halawany, A.M.; Abdallah, H.M.; Ibrahim, S.R.M. New xanthones and cytotoxic constituents from Garcinia mangostana fruit hulls against human hepatocellular, breast, and  J. Ethnopharmacol. 2017, 198CrossRef 19.Semwal, R.B.; Semwal, D.K.; Vermaak, I.; Viljoen, A. A comprehensive scientiﬁc overview of Garcinia  Fitoterapia 2015, 102CrossRef 20.Tousian Shandiz, H.; Razavi, B.M.; Hosseinzadeh, H. Review of Garcinia mangostana and its Xanthones in Metabolic Syndrome and Related Complications. Phytother. Res. 2017, 31CrossRef 21.Alam, F.; Islam, M.A.; Kamal, M.A.; Gan, S.H. Updates on Managing Type 2 Diabetes Mellitus with Natural Products: Towards Antidiabetic Drug Development. Curr. Med. Chem. 2018, 25CrossRef 22.Liu, Q.Y.; Wang, Y.T.; Lin, L.G. New insights into the anti-obesity activity of xanthones from Garcinia  Food Funct. 2015, 6CrossRef 23.Nguyen, C.N.; Trinh, B.T.D.; Tran, T.B.; Nguyen, L.T.; Jager, A.K.; Nguyen, L.D. Anti-diabetic xanthones from the bark of Garcinia xanthochymus. Bioorg. Med. Chem. Lett. 2017, 27CrossRefPubMed 24.Widowati, W.; Laksmitawati, D.R.; Wargasetia, T.L.; Aﬁfah, E.; Amalia, A.; Arinta, Y.; Rizal, R.; Suciati, T. Mangosteen peel extract (Garcinia mangostana L. ) as protective agent in glucose-induced mesangial cell as  Iran. J. Basic Med. Sci. 2018, 21PubMed 25.Li, Y.; Zhao, P.; Chen, Y.; Fu, Y.; Shi, K.; Liu, L.; Liu, H.; Xiong, M.; Liu, Q.H.; Yang, G.; et al. Depsidone and xanthones from Garcinia xanthochymus with hypoglycemic activity and the mechanism of promoting glucose uptake in L6 myotubes. Bioorg. Med. Chem. 2017, 25CrossRefPubMed 26.Maia-Landim, A.; Ramirez, J.M.; Lancho, C.; Poblador, M.S.; Lancho, J.L. Long-term eﬀects of Garcinia cambogia/Glucomannan on weight loss in people with obesity, PLIN4, FTO and Trp64Arg polymorphisms. BMC Complement. Altern. Med. 2018, 18CrossRef 27.Liu, S.; Li, D.; Huang, B.; Chen, Y.; Lu, X.; Wang, Y. Inhibition of pancreatic lipase, alpha-glucosidase, alpha-amylase, and hypolipidemic eﬀects of the total ﬂavonoids from Nelumbo nucifera leaves. J. Ethnopharmacol. 2013, 149CrossRef 28.Chen, J.J.; Chen, I.S.; Duh, C.Y. Cytotoxic xanthones and biphenyls from the root of Garcinia linii. Planta Med. 2004, 70CrossRef 29.Chen, J.J.; Peng, C.F.; Huang, H.Y.; Chen, I.S. Benzopyrans, biphenyls and xanthones from the root of Garcinia linii and their activity against Mycobacterium tuberculosis. Planta Med. 2006, 72CrossRef 30.Pantidos, N.; Boath, A.; Lund, V.; Conner, S.; McDougall, G.J. Phenolic-rich extracts from the edible seaweed, ascophyllum nodosum, inhibit  $\alpha$ -amylase and  $\alpha$ -glucosidase: Potential anti-hyperglycemic eﬀects. J. Funct. Foods 2014, 10CrossRef 31.Hardie, D.G.; Schaﬀer, B.E.; Brunet, A. AMPK: An Energy-Sensing Pathway with Multiple Inputs and Outputs. Trends Cell Biol. 2016, 26CrossRef 32. Biochem. Pharmacol. 2018, 153CrossRef 33.Scott, J.W.; Ling, N.; Issa, S.M.; Dite, T.A.; O’Brien, M.T.; Chen, Z.P.; Galic, S.; Langendorf, C.G.; Steinberg, G.R.; Kemp, B.E.; et al. Small molecule drug A-769662 and AMP synergistically activate naive AMPK independent of upstream kinase signaling. Chem. Biol. 2014, 21CrossRefPubMed 34.Oh-Hashi, K.; Irie, N.; Sakai, T.; Okuda, K.; Nagasawa, H.; Hirata, Y.; Kiuchi, K. Elucidation of a novel phenformin derivative on glucose-deprived stress responses in HT-29 cells. Mol. Cell. Biochem. 2016, 419, 29–40. [CrossRef] [PubMed] 35.Dujic, T.; Causevic, A.; Bego, T.; Malenica, M.; Velija-Asimi, Z.; Pearson, E.R.; Semiz, S. Organic cation transporter 1 variants and gastrointestinal side eﬀects of metformin in patients with Type 2 diabetes. Diabet. Med. 2016, 33CrossRefPubMed 36.Seo, K.I.; Choi, M.S.; Jung, U.J.; Kim, H.J.; Yeo, J.; Jeon, S.M.; Lee, M.K. Eﬀect of curcumin supplementation on blood glucose, plasma insulin, and glucose homeostasis related enzyme activities in diabetic db/db mice. Mol. Nutr. Food Res. 2008, 52CrossRefPubMed 37.Nazeri, S.; Farhangi, M.; Modarres, S. The eﬀect of diﬀerent dietary inclusion levels of rutin (a ﬂavonoid) on some liver enzyme activities and oxidative stress indices in rainbow trout, Oncorhynchus mykiss (Walbaum) exposed to Oxytetracycline. Aquac. Res. 2017, 48CrossRef 38.Meena, K.P.; Vijayakumar, M.R.; Dwibedy, P.S. Catechin-loaded Eudragit microparticles for the management of diabetes: Formulation, characterization and in vivo evaluation of antidiabetic eﬃcacy. J. Microencapsul. 2017, 34CrossRefPubMed 39.Che Hassan, N.K.N.; Taher, M.; Susanti, D. Phytochemical constituents and pharmacological properties of Garcinia xanthochymus—A review. Biomed. Pharmacother. 2018, 106CrossRef 40.Wang, Q.; Imam, M.U.; Yida, Z.; Wang, F. Peroxisome Proliferator-Activated Receptor Gamma (PPARgamma) as a Target for Concurrent Management of Diabetes and Obesity-Related Cancer. Curr. Pharm. Des. 2017, 23, 3677–3688. [CrossRef] 41.Janani, C.; Kumari, B.D.R. PPAR gamma gene—A review. Diabetes Metab. Syndr. 2015, 9CrossRef 42.Polvani, S.; Tarocchi, M.; Tempesti, S.; Bencini, L.; Galli, A. Peroxisome proliferator activated receptors at the  World J. Gastroenterol. 2016, 22CrossRef 43.Siersbaek, R.; Nielsen, R.; Mandrup, S. PPARgamma in adipocyte diﬀerentiation and metabolism–novel insights from genome-wide studies. FEBS Lett. 2010, 584CrossRefPubMed 44.Sebastiao, I.; Candeias, E.; Santos, M.S.; de Oliveira, C.R.; Moreira, P.I.; Duarte, A.I. Insulin as a Bridge between Type 2 Diabetes and Alzheimer Disease - How Anti-Diabetics Could be a Solution for Dementia. Front. Endocrinol. 2014, 5CrossRefPubMed 45.Elaidy, S.M.; Hussain, M.A.; El-Kherbetawy, M.K. Time-dependent therapeutic roles of nitazoxanide on high-fat diet/streptozotocin-induced diabetes in rats: Eﬀects on hepatic peroxisome proliferator-activated  Can. J. Physiol. Pharmacol. 2018, 96CrossRefPubMed 46.Sharma, K.; Kang, S.; Gong, D.; Oh, S.H.; Park, E.Y.; Oak, M.H.; Yi, E. Combination of Garcinia cambogia Extract and Pear Pomace Extract Additively Suppresses Adipogenesis and Enhances Lipolysis in 3T3-L1  Pharmacogn. Mag. 2018, 14 47.Bohm, A.; Wagner, R.; Machicao, F.; Holst, J.J.; Gallwitz, B.; Stefan, N.; Fritsche, A.; Haring, H.U.; Staiger, H. DPP4 gene variation aﬀects GLP-1 secretion, insulin secretion, and glucose tolerance in humans with high body adiposity. PLoS ONE 2017, 12CrossRef 48.Nahon, K.J.; Doornink, F.; Straat, M.E.; Botani, K.; Martinez-Tellez, B.; Abreu-Vieira, G.; van Klinken, J.B.; Voortman, G.J.; Friesema, E.C.H.; Ruiz, J.R.; et al. Eﬀect of sitagliptin on energy metabolism and brown adipose  Diabetologia 2018, 61CrossRef 49.Araujo, F.; Shrestha, N.; Gomes, M.J.; Herranz-Blanco, B.; Liu, D.; Hirvonen, J.J.; Granja, P.L.; Santos, H.A.; Sarmento, B. In vivo dual-delivery of glucagon like peptide-1 (GLP-1) and dipeptidyl peptidase-4 (DPP4)  Nanoscale 2016, 8 [CrossRef] 50.Andel, M.; Skrha, P.; Kraml, P.; Potockova, J.; Hoﬀmanova, I.; Silhova, E.; Fontana, J.; Richterova, A.; Gadiredi, M.; Busek, P.; et al. Annual monitoring of side eﬀects of administering sitagliptin in patients with type 2 diabetes mellitus. Vnitr. Lék. 2016, 62 51.Huang, P.K.; Lin, S.X.; Tsai, M.J.; Leong, M.K.; Lin, S.R.; Kankala, R.K.; Lee, C.H.; Weng, C.F. Encapsulation of 16-Hydroxycleroda-3,13-Dine-16,15-Olide in Mesoporous Silica Nanoparticles as a Natural Dipeptidyl Peptidase-4 Inhibitor Potentiated Hypoglycemia in Diabetic Mice. Nanomaterials 2017, 7CrossRef 52.Hayamizu, K.; Hirakawa, H.; Oikawa, D.; Nakanishi, T.; Takagi, T.; Tachibana, T.; Furuse, M. Eﬀect of Garcinia  Fitoterapia 2003, 74CrossRef 53.Hsu, C.-Y.; Shih, H.-Y.; Chia, Y.-C.; Lee, C.-H.; Ashida, H.; Lai, Y.-K.; Weng, C.-F. Rutin potentiates insulin receptor kinase to enhance insulin-dependent glucose transporter 4 translocation. Mol. Nutr. Food Res. 2014, 58CrossRefPubMed 54. ´Swiderska, E.; Strycharz, J.; Wróblewski, A.; Szemraj, J.; Drzewoski, J.; ´Sliwi´nska, A. Role of PI3K/AKT Pathway in Insulin-Mediated Glucose Uptake. Glucose Transport [Working Title] online: https://www.intechopen.com/online-ﬁrst/role-of-pi3k-akt-pathway-in-insulin-mediated-glucose- uptake (accessed on 23 October 2019). [CrossRef] 55.Hsu, C.Y.; Sulake, R.S.; Huang, P.K.; Shih, H.Y.; Sie, H.W.; Lai, Y.K.; Chen, C.; Weng, C.F. Synthetic -antroquinonol exhibits dual actions against insulin resistance by triggering AMP kinase and inhibiting $(+)$   Br. J. Pharmacol. 2015, 172CrossRefPubMed 56.Rebollo-Hernanz, M.; Zhang, Q.; Aguilera, Y.; Martin-Cabrejas, M.A.; de Mejia, E.G. Phenolic compounds from coﬀee by-products modulate adipogenesis-related inﬂammation, mitochondrial dysfunction, and insulin resistance in adipocytes, via insulin/PI3K/ Food Chem. Toxicol. 2019, 132, 110672. [CrossRef] [PubMed] 57.Taher, M.; Zakaria, T.M.F.S.T.; Susanti, D.; Zakaria, Z.A. Hypoglycaemic activity of ethanolic extract of  BMC Complement. Altern. Med. 2016, 16 58.Asghar, M.; Monjok, E.; Kouamou, G.; Ohia, S.E.; Bagchi, D.; Lokhandwala, M.F. Super CitriMax (HCA-SX) attenuates increases in oxidative stress, inﬂammation, insulin resistance, and body weight in developing obese Zucker rats. Mol. Cell. Biochem. 2007, 304CrossRef 59.Talpur, N.; Echard, B.W.; Yasmin, T.; Bagchi, D.; Preuss, H.G. Eﬀects of niacin-bound chromium, Maitake mushroom fraction SX and (-)-hydroxycitric acid on the metabolic syndrome in aged diabetic Zucker fatty rats. Mol. Cell. Biochem. 2003, 252CrossRef  by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access $\copyright2019$  article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

                                                   

                                                  

                                                 

                                                

                                               

                                              

                                             

                                            

                                           

                                          

                                         

                                        

                                       

                                      

                                     

                                    

                                   

                                  

                                 

                                

                               

                              

                             

                            

                           

                          

                         

                        

                       

                      

                     

                    

                   

                  

                 

                

               

              

             

            

           

          

         

        

       

      

     

    

   

  

 


Gabriel Morales-Cid Soledad Ca´ rdenas Bartolome´ M. Simonet Miguel Valca´ rcel Department of Analytical Chemistry, Campus de Rabanales, University of Cordoba, Cordoba, Spain Received July 29, 2008 Revised October 23, 2008 Accepted October 26, 2008

 



# Research Article

# Direct automatic determination of free and total anesthetic drugs in human plasma by use of a dual (microdialysis–microextraction by packed sorbent) sample treatment coupled at-line to NACE–MS

This paper reports for the ﬁrst time the use of microextraction by packed sorbent in combination with CE. The combined system was used to determine anesthetic drugs in human plasma. A microdialysis ﬁber was coupled on-line to the microextraction unit in order to distinguish between free and total concentrations of drugs. The system was automated by connecting the microextraction unit to a syringe pump and interfacing it to a computer. The ensuing method allows the determination of  $10\,\upmu\mathrm{g/L}$  concentrations of free drugs and  $1\,\upmu\mathrm{g/L}$  concentrations of total drugs from only  $200\,\upmu\mathrm{L}$  of sample with an RSD of less than  $9\%$ .

### Keywords: Anesthetics  $/$  Human plasma / Microdialysis / Microextraction by packed sorbent / NACE-MSDOI 10.1002/elps.200800493

## 1Introduction

Obtaining quality analytical information in pharmacology entails minimizing sample preparation and analysis times. Sample treatments are especially important in this ﬁeld owing to the high complexity of some sample matrices such as plasma and urine. Sample preparation should be simpliﬁed by using automated devices and expeditious, environmentally friendly, inexpensive methodologies invol- ving reduced human intervention. In addition, procedures shouldbehighlyreproducible,providehighanalyte recoveriesandfacilitateenrichmentwiththetarget compounds.

This work was undertaken with the aim of simplifying sample treatment for the determination of local anesthetics such as lidocaine, mepivacaine and bupivacaine in human plasma, which should be administered in the minimal dose required by each individual patient according to age, weight and health status.

Solid-phase extraction (SPE) is probably the most widely used sample preparation technique by virtue of its affording concentration and cleanup of target compounds from a complex sample matrix. Among other applications, SPE has been used in combination with HPLC for the extraction of local anesthetics in human plasma [1–7]. Although it usually provides good recoveries  $(\sim\!70\%)$  of sorbent (more than  $100\,\mathrm{mg})$  and also large sample volumes  $(\geq1\,\mathrm{mL})$  been used for the extraction of local anesthetics in plasma samples [9–11]; this technique, however, uses large sample volumes  $(\geq1\,\mathrm{mL})$  , provides low recoveries and has the additional $(>40\,\mathrm{min})$  disadvantage of the short useful life of the ﬁbers.



Microextrationbypackedsorbent(MEPS),which is also known as ‘‘microextraction by packed syringe’’, integrates a miniaturized SPE bed with a microvolume syringe( $.100{-}250\,\upmu\mathrm{L})$ .CommercialMEPSequipment uses an integrated barrel insert and needle device to reduce SPE to a microscale suitable for small volume samples and for the on-line adaptation of conventional SPEtechniques.Inaddition,MEPSequipmentis portable and can be operated manually or connected to a robotic sampler [12]. MEPS features more expeditious sample preparation and uses organic solvents more spar- ingly than both liquid–liquid extraction and SPE [13]. In fact, MEPS minimizes sample volumes, reduces sample preparation times and solvent volumes, and combines sample extraction and sample injection, which is done directly in LC and GC. Among others, MEPS has been used in combination with LC–MS [8, 14–16], LC-MS/MS [17–21] and GC-MS [8, 15, 21, 22] for the determination of drugs in However, MEPS has never previously been used in combination with CE. Although the CE technique features a high resolution power, it has limited sensitivity, even when coupled to an MS detector. This requires using some sample concentration pretreatment in order to ensure acceptable analytical performance. The small sample volumes used in CE (a few nanoliters only) can be highly advantageous with a view to maximizing analyte enrichment by using a well- designed sample treatment. Therefore, combining MEPS and CE can provide interesting advantages. Local anesthetics and their metabolites have so far been determined by CZE [23–28], EKC [12, 29–32] and NACE [26, 33–38].



The purpose of this work was to assess the potential of the at-line coupling of MEPS to CE–MS for the determina- tion of free and total concentrations of local anesthetics in human plasma. The coupling of the MEPS-CE/MS combi- nation in-line to microdialysis ﬁber is also described in the paper to perform the rapid and automatic analysis of free drugs.

## 2Materials and methods

### 2.1 Reagents and standards

Lidocaine,mepivacainehydrochlorideandbupivacaine hydrochloride (Sigma–Aldrich; Madrid, Spain) were used as standards. Ammonium acetate, formic acid (Sigma– Aldrich;), methanol and acetonitrile (Panreac Quı´mica, S. A.; Barcelona, Spain) were used to prepare the BGE. Sodium hydroxide and hydrochloric acid (Panreac Quı´mica) were used for  $\mathrm{pH}$  adjustment and capillary conditioning. Finally, trichloroacetic acid and sodium chloride (Panreac Quı´mica) were used for sample treatment.

Standard stock solutions of the target analytes were prepared by dissolving in methanol or water the amount requiredtoobtaina $50\,\mathrm{mg/L}$ concentrationofeach compound and stored at  $4^{\circ}C$  until use. An acid  $\mathrm{pH}$  was required to dissolve the analytes in water, which was thus supplied with  $1\%$  formic acid for this purpose. Standard solutions of appropriate concentrations were prepared on a daily basis by diluting the stock solutions in methanol.

### 2.2 Instrumentation

An Agilent  $\mathrm{HP}^{3\mathrm{D}}$  Capillary Electrophoresis system (Wald- bronn, Germany) coupled to an Agilent 1100 Series LC/ MSD mass spectrometer via an electrospray-API interface was used. The make-up ﬂow of sheath liquid was delivered by the Agilent 1100 isocratic pump, which was operated at a 1:100 split ratio. The uncoated fused-silica capillaries used  $(80\,\mathrm{cm}\times50\,\upmu\mathrm{m}$  $375\,\upmu\mathrm{m}$  od) were supplied by Beckman (Fullerton, CA). The CE-MS system was controlled, and data wereacquiredandprocessed,byusingtheAgilent ChemStation software.

### 2.3 Instrumental operating conditions

The BGE employed was previously used elsewhere [38]. Each new capillary was initially conditioned with  $1\,\mathrm{M\,\,HCl}$  (5 min), 0.1 M NaOH (10 min) and Milli- $\mathsf Q$  water (5 min), in this sequence. Each day, the capillary in use was conditioned with 0.1 M NaOH ( $(10\,\mathrm{min})$ , Milli-Q water  $(5\,\mathrm{min})$  and running buffer  $(5\,\mathrm{min})$ . Between runs, the capillary was rinsed with MeOH (3 min) and BGE (6 min). All condition- ing steps were performed at 2 bar.

The optimum BGE was found to consist of  $2\,\mathrm{M}$  formic acid and  $70\,\mathrm{mM}$  ammonium acetate in 60:40 acetoni- trile:methanol. The capillary cartridge was kept at  $20^{\circ}C$  The actual capillary temperature may have varied by a few degrees depending on the room temperature level by effect of the CE equipment being coupled to the mass spectro- meter. As recommended, the sheath liquid consisted of an  $80{:}20\,\mathrm{v/v}$  methanol–water mixture containing  $2\%$  formic acid and was used at a ﬂow rate of  $4\,\upmu\mathrm{L}/\mathrm{min}$ .

Samples were kept at ambient temperature in the autosampler and injected in the hydrodynamic mode at 50 mbar for 5 s, followed by running buffer, also at 50 mbar, for 3 s. Tests were carried out in the positive polarity mode, using a constant voltage of  $25\,\mathrm{kV}$ .

The ESI and MS operating conditions were as follows: fragmenter voltage,  $70\,\mathrm{V}$ ; nebulizer pressure, 1 psi;  $\mathrm{N}_{2}$  (drying gas) ﬂow rate, 3 L/min;  $\Nu_{2}$  (drying gas) temperature,  $150^{\circ}\mathrm{C}_{\mathrm{i}}$  monitoring the ions with the  $m/z$  values corresponding to . Such values were 235.2 for lidocaine, 247.2 for ${[\mathrm{{M}}+\mathrm{{H}}]}^{+}$  mepivacaine and 289.2 for bupivacaine. Quantiﬁcation was based on the corresponding ion for each individual analyte. The spray chamber was cleaned with methanol and Milli- $\mathsf Q$  water on a weekly basis. Also, daily cleaning with Milli- $\mathsf Q$  water is recommended.

### 2.4 Automatic setup for sample preconcentration

Sample treatment was performed with a  $200\,\upmu\mathrm{L}$  syringe (SGE Analytical Science) ﬁtted to a microextraction by packed sorbents system (MEPS) (SGE Analytical Science); each barrel insert and needle contained  ${\sim}4\,\mathrm{mg}$  of C18 packing of particle size  $45\,\upmu\mathrm{m}$  and pore size  $60\,\mathrm{\AA}$ .

The system was automated by connecting the syrin- ge–MEPS assembly to a syringe pump (Cavro) that was controlled via a computer running the software Sagittarius 3.0, which was obtained from the working group on Chemical Analysis and Vibrational Spectroscopy of the Vienna University of Technology, Austria.

For the analysis of free drugs, a microdialysis probe was on-line connected to the needle of the syringe-MEPS system. The microdialysis probe was a CMA 20 Elite Microdialysis model from CMA/Microdialysis AB (Stockholm, Sweden) withthefollowingspeciﬁcations:membrane,poly- arylethersulfone; membrane length,  $10\,\mathrm{mm}_{\mathrm{i}}$  membrane od,  $0.5\,\mathrm{mm}_{\mathrm{i}}$ ; cutoff,  $20\,000\,\mathrm{Da}$ ; shaft length,  $14\:\mathrm{mm}$ .

The MEPS system was conditioned with methanol ﬁrst and then water prior to ﬁrst use. In addition, it was washed with  methanol after each run in order to minimize carryover. $100\,\upmu\mathrm{L}$ 

### 2.5 Sample preparation

Human plasma and serum samples were centrifuged at  $4500\,\mathrm{rpm}$  for 5 min. The sample preparation procedure used to determine free concentrations of the target analytes in human plasma or serum involved adding  $40\,\mathrm{g/L}$  NaCl to  of sample and adjustment to pH 7.4. On the other $200\,\upmu\mathrm{L}$  hand, determinations of total analyte concentrations required that samples be treated as follows: mixtures containing 250 or  of sample and 50 or  $100\,\upmu\mathrm{L}$  $500\,\upmu\mathrm{L}$  of trichloroacetic acid were centrifuged at  $5000\,\mathrm{rpm}$  for 15 min [39], the supernatants being collected and adjusted to  $\mathrm{pH}~7.4\$  for analysis.

## 3Results and discussion

The structures of target analytes, three local anesthetics, are shown in Fig. 1. The results obtained in the determination of their free and total concentrations in human plasma are presented and discussed below.

### 3.1 NACE separation and MS detection

Based on the compatibility of the BGEs involved with MS detection, we chose to use NACE on the grounds that it avoids the use of a BGE with high capillary current intensity. Initially, we used an MeOH-based BGE. However, due to the strong EOF resulting from the low viscosity and high dielectric constant of the alcohol the addition of acetonitrile was required in order to improve selectivity and to reduce analysis times. Acetonitrile contents in the mixture up to  $60\%$  resulted in markedly improved baseline stability and resolution; on the other hand, contents above  $60\%$  raised the current intensity without increasing resolution. Therefore, a  $60{\cdot}40\,\mathrm{v}/\mathrm{v}$  acetoni- trile–methanol mixture was selected for further testing. We chose to use ammonium acetate at concentrations ranging from 35 to  $80\,\mathrm{mM}$  as electrolyte. The results thus obtained suggested that a  $70\,\mathrm{mM}$  concentration sufﬁced to ensure a stable current intensity and good separation. The effect adding formic acid to the BGE was also examined. Using an acid medium would cause the analytes to be protonated and facilitate their separa- tion as cations. Improving separation to a substantial extent required adding a 2 M concentration of formic acid, which had little effect on the current intensity. Injection was performed in the hydrodynamic mode by applying 50 mbar for 5 s, followed by BGE, also at 50 mbar, for 3 s. Increasing the sample injection time beyond the recommended value resulted in current interruption, so it was discarded. The effect of the applied voltage on the separation was examined over the range  $15{-}25\,\mathrm{kV}.$  Neither was any Joule effect observed nor were any exceedingly high current intensity values obtained over such a range, so a voltage of  $25\,\mathrm{kV}$  was adopted.



Spray chamber temperatures from  $120–200^{\circ}\mathsf{C}$  were studied.Thesensitivitywasfoundtodecreasewith increasing chamber temperature. On the other hand, reso- lution improved as the temperature was raised. A compro- mise must therefore be made between these two analytical properties. A temperature of  $150^{\circ}C$ resulted in only slightly reduced sensitivity, but markedly improved resolution, so it was chosen for further testing. The sheath liquid composi- tion is known to affect both electrospray stability and the ionization process. We studied various mixtures of metha- nol, acetonitrile, water and formic acid. Based on the results, an  $80{:}20\,\mathrm{v/v}$ methanol–water mixture containing  $2\%$  formic acid provided the best results in terms of electric contact and sensitivity. Sheath liquid ﬂow rates ranging from 2 to  $6\,\upmu\mathrm{L}/\upmu$  min were studied. The highest ﬂow rate resulted in lower sensitivity than the lowest one. A  $4\,\upmu\mathrm{L}/\mathrm{min}$  value was adopted as optimal since it provided better sensitivity than the lowest rate.

The nebulizer pressure has a critical inﬂuence on resolution. Pressure values from 1 to 6 psi were studied here. Pressures above 4 psi were found to result in current interruptions. Resolution was better at the lower pressures by effect of a reduced Venturi effect of the nebulizing gas at the capillary tip. Therefore, a pressure of 1 psi was selected as optimal. We also examined the inﬂuence of the ﬂow rate of drying gas, which was studied over the range  $1{-}6\;\mathrm{L}/\mathrm{min};$  based on the results, the best sensitivity was achieved with  $3\;\mathrm{L/min}$ . Finally, the inﬂuence of the fragmenter voltage on the ionization process was studied over the range 60–120 V and a value of  $70\,\mathrm{V}$  was selected.

### 3.2 MEPS preconcentration

The free fraction of the proposed analytes in human plasma was analyzed by using the microdialysis probe as coupled to MEPS. The microdialysis probe allows the easy separation of the free analytes from the analytes bounded to proteins that cannot pass through the microdialysis membrane. Preli- minary MEPS tests were performed on water samples spiked with a  $100\,\upmu\mathrm{g/L}$  concentration of each standard; then, an aliquot of  $200\,\upmu\mathrm{L}$  was preconcentrated at  $10\,\upmu\mathrm{L}/\mathrm{s}$ . Before  at $(2\times200\,\upmu\mathrm{L}$   and then eluted with methanol four times $100\,\upmu\mathrm{L}/\mathrm{s})$  . The analysis of the four consecutive extractions $(20\,\upmu\mathrm{L}\times4)$  revealed the presence of traces of the analytes in the second and third eluates, but none in the fourth. Based on these results, we reduced the analyte concentrations down to . However, the consecutive eluates still contained the $5\,\upmu\mathrm{g/L}$  analytes in proportions of  $29.3{-}42.3~\%$  with respect to the ﬁrst. This led us to add formic acid to the eluent in order to facilitate quantitative elution. The formic acid altered the distribution of the analytes in the sorbent and eluent. The acid concentrations studied ranged from 5 to  $200\,\mathrm{mM}$  (see Fig. 2). With  $10\,\mathrm{mM}$  formic acid, the second eluate exhibited  $9.8–20\%$  carryover. Based on the results, using  $20\,\mathrm{mM}$  concentration sufﬁced to ensure quantitative elution of the analytes since carryover was reproducible and always less than  $6\%$ .



Then, the effect of  $\mathrm{pH}$  was examined over the range 3.6–9.5, where no signiﬁcant changes in sensitivity were observed. Therefore,  $\mathrm{pH}~7.4$  was chosen in order to avoid differences with plasma  $\mathrm{pH}$ .

The RSD obtained for six spiked water samples under these conditions was high. This was ascribed to errors arising during the analytical procedure (particularly from human manipulation, the low volume of eluent used and capillary conditioning). In order to improve repeatability, we increased the conditioning time from 3 to  $6\,\mathrm{min}$ , which resulted in dramatically reduced RSD values. Also, increas- ing the eluent volume from 20 to  $50\,\upmu\mathrm{L}$  resulted in further improved RSD. Finally, using a syringe pump substantially reduced the need for human intervention and its associated errors as a result. In addition, incorporation of the syringe pump enabled the use of small eluent volumes (down to ) with good RSD values. $25\,\upmu\mathrm{L}$  The above-described procedure was used to examine the effect of coupling the microdialysis probe via a small piece  $(0.5\,\mathrm{cm})$  of Teﬂon tubing. Figure 3 illustrates the coupling of the automated MEPS system by using a syringe pump withacoupledmicrodialysisprobe.Theprobewas immersed in the sample, held in a vial furnished with a glass insert. As expected, preliminary tests showed that the ﬁber was permeable to the target analytes. However, using the above-described procedure unaltered resulted in strongly diminishedsensitivity.Inmicrodialysis,theabsolute recovery (mol/time unit) of a compound from the sample depends on (i) the ‘‘cut-off’’ of the dialysis membrane, (ii) the length of the membrane, (iii) the ﬂow rate of the and  $10\;S_{\mathrm{y/x}}/$ slope, respectively. The former ranged from 10.4 to  $15.2\,\upmu\mathrm{g/L}$  for free concentrations and 0.6 to  $1.6\,\upmu\mathrm{g/L}$  for total concentrations. The RSD values for peak areas and migration times were under 9 and  $1.4\%,$ , respectively, for both methods. As can be seen from the table, preconcentra- tion took  $35\,\mathrm{min}$  with MEPS-microdialysis and only  $2\,\mathrm{min}$  with the MEPS procedure. LODs were markedly better with MEPS than with microdialysis and MEPS in combination (viz. 0.6 versus  $10.4\,\upmu\mathrm{g/L})$ . This testiﬁes to the better suitability of MEPS than microdialysis–MEPS for the determination of total analyte concentrations.



perfusion ﬂuid and (iv) the diffusion coefﬁcient of the compound through the ﬂuid. Because the former two variables had previously been examined and adjusted, we focused on the latter two. First, we examined the effect of the ﬂow rate of perfusion ﬂuid (Milli-Q water) and hence the contact time between the sample and perfusion ﬂuid, and found increasing the contact time between the sample and ﬁber to boost diffusion of the analytes through the perfusion ﬂuid. The contact time could be improved by holding the ﬂow sequentially or by reducing the ﬂow rate. We performed several tests in order to study the effect of the contact time between sample and ﬁber. The best diffusion rates were obtained with sequential loadings of  $5\,\upmu\mathrm{L}$ at  $0.1\,\upmu\mathrm{L}/\mathrm{s}\!+\!60\,\mathrm{s}$  waiting time 40 times. The total sample volume passed through the sorbent was thus  $100\,\upmu\mathrm{L}.$  However, a long time was required that was successfully shortened by using a continuous sequence without breaks at 0.1 mL/s and then repeating the process another time (100 mL 2).

The next step involved modifying the ionic strength of the sample media. For this purpose, NaCl at concentrations ranging from 10 to 200 g/L was added to the samples. The best recov- eries were obtained with 40 g/L. On the other hand, concen- trations above 200 g/L resulted in ﬁber caking and clogging.

The sample treatment used for the determination of total analyte concentrations in human plasma is described in Section 2. The supernatant collected from the plasma follow- ing treatment with trichloroacetic acid and centrifugation was adjusted to pH 7.4 and preconcentrated with MEPS at 2 mL/s, which was slow enough to ensure optimum analyte retention in the C18 bed. Elution was done with 25 mL of eluent. As stated above, this volume was large enough to ensure repetitive results (RSD ranged from 4.5 to 7.6%).

### 3.4 Analysis of human plasma

a) In these samples, the loaded volume was reduced down to . $250\,\upmu\mathsf{L}$  To demonstrate this interaction with proteins, several tests were done on human serum as sample because of their low content in proteins. As can be seen in Table 2, the recoveries for lidocaine, mepivacaine and bupivacaine spanned the ranges 34.4–37.7, 44.7–46 and  $11.7{-}12.7\%$ , respectively. As expected, the recoveries from serum samples exceeded those from plasma samples by effect of the former containing less protein and a larger free fraction of analytes as a result.



Table 3 shows the ﬁgures of merit of the determination of total analyte concentrations in different plasma samples. As can be seen, samples 1, 6 and 7 tested positive for lido- caine. In addition, analytes recoveries were determined in four different samples spiked with a  $10\,\upmu\mathrm{g/L}$  concentration of each analyte. The estimated recoveries ranged from 78 to  $92\%$  for lidocaine, 58.39 to  $68.7\%$  for mepivacaine and 42.5 to  $56.59\%$  for bupivacaine. As expected, drug recoveries were higher than with the free concentration method by effect of bonds with proteins being broken and the resulting free drugs joining the free fraction. We should note that a volume of sample as low as  $250\,\upmu\mathrm{L}$  should sufﬁce for the intended purpose since samples 6 and 7 exhibited no signiﬁcant difference from sample 1, which was analyzed by using  $500\,\upmu\mathrm{L}$ .



The electropherograms of Fig. 4 expose the differences between the free concentration method and the total concentration method. Figure 4A corresponds to an aliquot of a single plasma sample analyzed with the free concen- tration method, both unspiked (1) and spiked with  $1.6\,\mathrm{mg/L}$  lidocaine, and  $1.4\,\mathrm{mg/L}$  mepivacaine and bupivacaine (2).

Figure 4B shows the electropherograms obtained in the total concentration analysis of other aliquots of the same plasma sample, both unspiked (1) and spiked with  $5\,\upmu\mathrm{g/L}$  lidocaine, and  $4.4\,\upmu\mathrm{g/L}$  mepivacaine and bupivacaine (2).

Figure5showstwoselectedelectropherograms obtained by using the total concentration method. Finally, Fig. 5 (1) corresponds to an unspiked human plasma sample aliquot, which tested positive for lidocaine, and Fig. 5 (2) to an aliquot of the same sample spiked with up to  lidocaine. $10\,\upmu\mathrm{g/L}$ 

## 4Concluding remarks

The high potential of the combination MEPS-CE has been demonstrated. MEPS provides the efﬁcient preconcentration of analytes from a small sample volume. In addition, its automatization has been achieved by using a simple syringe pump. The use of a microdialysis needle on-line coupled to the MEPS-syringe had permitted the analysis of free and total drug in human plasma. This simple strategy is then recommended to perform biological studies.

The authors are grateful to Spain’s Ministry of Education and Science for funding this work as a part of Project 2007- 60426. G.M.-C. also wishes to thank the Ministry for the award of a Research Training Fellowship (Grant BES-2005-10603).

The authors have declared no conﬂict of interest.

## 5References

[1] Drewe, J., Rufer, S., Huwyler, J., Ku¨ sters, E., matogr. B 1997, 691, 105–110

[2] Abdel-Rehim, M., Bielenstein, M., Askemark, Y., Tyre- fors, N., Arvidsson, T., J. Chromatogr. B 2000, 741, 175–188.

[3] Kang, L ., Jun, H. W., McCall, J. W., J. Pharm. Biomed. Anal. 1999, 19, 737–745.

[4] Reif, S., Le Corre, P., Dollo, G., Chevanne, F., Le Verge, R., J. Chromatogr. B 1998, 719, 239–244.

[5] Arvidsson, T., Eklund, E., J. Chromatogr. B 1995, 668, 91–98.

[6] Arvidsson, T., Askemark, Y., Halldin, M. M., Biomed. Chromatogr. 1999, 13, 286–292.

[7] Rifai, N., Hsin, O., Hope, T., Sakamoto, M., Ther. Drug Monit. 2001, 23, 182–186.

[8] Altun, Z., Abdel-Rehim, M., Blomberg, L. G., J. Chro- matogr. B 2004, 813, 129–135.

[9] Abdel-Rehim, M., Andersson, M., Portelius, E., Norsten- Ho¨ o¨ g, C., Blomberg, L.G.,, 313–321.

[10] Koster, E. H. M., Wemes, C., Morsink, J. B., de Jong, G. J., J. Chromatogr. B 2000, 739, 175–182.

[11] Abdel-Rehim,M.,Bielenstein,M.,Arvidsson,T., J. Microcol. Sep. 2000, 12, 308–315.

[12] Wynne, P., Hibbert, R., DiFeo, D., Dawes, P., The Column. 2008, 4, 12– 17.

[13] El-Beqqali, A., Abdel-Rehim, M., J. Sep. Sci. 2007, 30, 2501–2505.

[14] Said, R., Hassan, Z., Hassan, M., Abdel-Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2008, 31, 683–694.

[15] Abdel-Rehim, M., Altun, Z., Blomberg, L., J. Mass Spectrom. 2004, 39, 1488–1493.

[16] Abdel-Rehim, M., Hassan, Z., Skansem, P., Hassan, M., J. Liq. Chromatogr. Rel. Technol. 30, 3029–3041.

[17] Abdel-Rehim, M., Skansen, P., Vita, M., Hassan, Z., Blomberg, L., Hassan, M., Anal. Chim. Acta 2005, 539, 35–39.

[18] Vita, M., Skansen, P., Hassan, M., Abdel-Rehim, M., J. Chromatogr. B 2005, 817, 303–307.

[19] Abdel-Rehim, M ., Andersson, L. I., Altun, Z., Blomberg, L. G., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 1725–1736.

[20] El-Beqqali,A.,Kussak,A.,Blomberg,L.,Abdel- Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2007, 30, 575–586.

[21] Abdel-Rehim, M., Dahlgren, M., Blomberg, L., Claude, S., Tabacchi, R., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 2537–2544.

[22] Abdel-Rehim, M ., J. Chromatogr. B 2004, 801, 317–321.

[23] Al-Nouti, Y., Bartlett, M. G., J. Am. Soc. Mass Spectrom. 2002, 13, 928–935.

[24] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[25] Chee, G. L., Wan, T. S. M., J. Chromatogr. B 1993, 612, 172–177.

[26] Wang, Z., Wan, H., Anderson, M. S., Abdel-Rehim, M., Blomberg, L. G., Electrophoresis 2001, 22, 2495–2502.

[27] Geiser, L., Rudaz, S., Veuthey, J. L., Electrophoresis 2003, 24, 3049–3056.

[28] Curcuruto, O., Zaramella, A., Hamdan, M., Turrina, S., Tagliaro, F., Rapid Commun. Mass Spectrom. 1995, 9, 1487–1491.

[29] Lurie, I. S., Hays, P. A., Parker, K., Electrophoresis 2004, 25, 1580–1591.

[30] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[31] Anderson, M. S., Wan, H., Abdel-Rehim, M., Blomberg, L. G., J. Microcol. Sep. 1999, 11, 620–626.

[32] Bretnall, A. E., Clarke, G. S., J. Chromatogr. A 1996, 745, 145–154.

[33] Bjørnsdottir, I ., Hansen, S. H., J. Biochem. Biophys. Methods 1999, 38, 155–161.

[34] Bjørnsdottir, I., Tjørrnelund, J., Hansen, S. H., Electro- phoresis 1998, 19, 2179–2186.

[35] Cherkaoui, S., Geiser, L., Veuthey, J. L., Chromato- graphia 2000, 52, 403–407.

[36] Zhang, J., Lou, Y., Hoogmartens, J., Van Schepdael, A., Electrophoresis 2006, 27, 4827–4835.

[37] Cherkaoui,S.,Veuthey,J.L.,Analysis1999,27, 765–771.

[38] Anderson, M. S., Lu, B., Abdel-Rehim, M., Blomberg, S., Blomberg, L. G., Rapid Commun. Mass Spectrom. 2004, 18, 2612–2618.

[39] March, J., Simonet, B. M., Grases, F., J. Chromatogr. B 2001, 757, 247–255.
Gabriel Morales-Cid Soledad Ca´ rdenas Bartolome´ M. Simonet Miguel Valca´ rcel Department of Analytical Chemistry, Campus de Rabanales, University of Cordoba, Cordoba, Spain Received July 29, 2008 Revised October 23, 2008 Accepted October 26, 2008

 



# Research Article

# Direct automatic determination of free and total anesthetic drugs in human plasma by use of a dual (microdialysis–microextraction by packed sorbent) sample treatment coupled at-line to NACE–MS

This paper reports for the ﬁrst time the use of microextraction by packed sorbent in combination with CE. The combined system was used to determine anesthetic drugs in human plasma. A microdialysis ﬁber was coupled on-line to the microextraction unit in order to distinguish between free and total concentrations of drugs. The system was automated by connecting the microextraction unit to a syringe pump and interfacing it to a computer. The ensuing method allows the determination of  $10\,\upmu\mathrm{g/L}$  concentrations of free drugs and  $1\,\upmu\mathrm{g/L}$  concentrations of total drugs from only  $200\,\upmu\mathrm{L}$  of sample with an RSD of less than  $9\%$ .

### Keywords: Anesthetics  $/$  Human plasma / Microdialysis / Microextraction by packed sorbent / NACE-MSDOI 10.1002/elps.200800493

## 1Introduction

Obtaining quality analytical information in pharmacology entails minimizing sample preparation and analysis times. Sample treatments are especially important in this ﬁeld owing to the high complexity of some sample matrices such as plasma and urine. Sample preparation should be simpliﬁed by using automated devices and expeditious, environmentally friendly, inexpensive methodologies invol- ving reduced human intervention. In addition, procedures shouldbehighlyreproducible,providehighanalyte recoveriesandfacilitateenrichmentwiththetarget compounds.

This work was undertaken with the aim of simplifying sample treatment for the determination of local anesthetics such as lidocaine, mepivacaine and bupivacaine in human plasma, which should be administered in the minimal dose required by each individual patient according to age, weight and health status.

Solid-phase extraction (SPE) is probably the most widely used sample preparation technique by virtue of its affording concentration and cleanup of target compounds from a complex sample matrix. Among other applications, SPE has been used in combination with HPLC for the extraction of local anesthetics in human plasma [1–7]. Although it usually provides good recoveries  $(\sim\!70\%)$  of sorbent (more than  $100\,\mathrm{mg})$  and also large sample volumes  $(\geq1\,\mathrm{mL})$  been used for the extraction of local anesthetics in plasma samples [9–11]; this technique, however, uses large sample volumes  $(\geq1\,\mathrm{mL})$  , provides low recoveries and has the additional $(>40\,\mathrm{min})$  disadvantage of the short useful life of the ﬁbers.



Microextrationbypackedsorbent(MEPS),which is also known as ‘‘microextraction by packed syringe’’, integrates a miniaturized SPE bed with a microvolume syringe( $.100{-}250\,\upmu\mathrm{L})$ .CommercialMEPSequipment uses an integrated barrel insert and needle device to reduce SPE to a microscale suitable for small volume samples and for the on-line adaptation of conventional SPEtechniques.Inaddition,MEPSequipmentis portable and can be operated manually or connected to a robotic sampler [12]. MEPS features more expeditious sample preparation and uses organic solvents more spar- ingly than both liquid–liquid extraction and SPE [13]. In fact, MEPS minimizes sample volumes, reduces sample preparation times and solvent volumes, and combines sample extraction and sample injection, which is done directly in LC and GC. Among others, MEPS has been used in combination with LC–MS [8, 14–16], LC-MS/MS [17–21] and GC-MS [8, 15, 21, 22] for the determination of drugs in However, MEPS has never previously been used in combination with CE. Although the CE technique features a high resolution power, it has limited sensitivity, even when coupled to an MS detector. This requires using some sample concentration pretreatment in order to ensure acceptable analytical performance. The small sample volumes used in CE (a few nanoliters only) can be highly advantageous with a view to maximizing analyte enrichment by using a well- designed sample treatment. Therefore, combining MEPS and CE can provide interesting advantages. Local anesthetics and their metabolites have so far been determined by CZE [23–28], EKC [12, 29–32] and NACE [26, 33–38].



The purpose of this work was to assess the potential of the at-line coupling of MEPS to CE–MS for the determina- tion of free and total concentrations of local anesthetics in human plasma. The coupling of the MEPS-CE/MS combi- nation in-line to microdialysis ﬁber is also described in the paper to perform the rapid and automatic analysis of free drugs.

## 2Materials and methods

### 2.1 Reagents and standards

Lidocaine,mepivacainehydrochlorideandbupivacaine hydrochloride (Sigma–Aldrich; Madrid, Spain) were used as standards. Ammonium acetate, formic acid (Sigma– Aldrich;), methanol and acetonitrile (Panreac Quı´mica, S. A.; Barcelona, Spain) were used to prepare the BGE. Sodium hydroxide and hydrochloric acid (Panreac Quı´mica) were used for  $\mathrm{pH}$  adjustment and capillary conditioning. Finally, trichloroacetic acid and sodium chloride (Panreac Quı´mica) were used for sample treatment.

Standard stock solutions of the target analytes were prepared by dissolving in methanol or water the amount requiredtoobtaina $50\,\mathrm{mg/L}$ concentrationofeach compound and stored at  $4^{\circ}C$  until use. An acid  $\mathrm{pH}$  was required to dissolve the analytes in water, which was thus supplied with  $1\%$  formic acid for this purpose. Standard solutions of appropriate concentrations were prepared on a daily basis by diluting the stock solutions in methanol.

### 2.2 Instrumentation

An Agilent  $\mathrm{HP}^{3\mathrm{D}}$  Capillary Electrophoresis system (Wald- bronn, Germany) coupled to an Agilent 1100 Series LC/ MSD mass spectrometer via an electrospray-API interface was used. The make-up ﬂow of sheath liquid was delivered by the Agilent 1100 isocratic pump, which was operated at a 1:100 split ratio. The uncoated fused-silica capillaries used  $(80\,\mathrm{cm}\times50\,\upmu\mathrm{m}$  $375\,\upmu\mathrm{m}$  od) were supplied by Beckman (Fullerton, CA). The CE-MS system was controlled, and data wereacquiredandprocessed,byusingtheAgilent ChemStation software.

### 2.3 Instrumental operating conditions

The BGE employed was previously used elsewhere [38]. Each new capillary was initially conditioned with  $1\,\mathrm{M\,\,HCl}$  (5 min), 0.1 M NaOH (10 min) and Milli- $\mathsf Q$  water (5 min), in this sequence. Each day, the capillary in use was conditioned with 0.1 M NaOH ( $(10\,\mathrm{min})$ , Milli-Q water  $(5\,\mathrm{min})$  and running buffer  $(5\,\mathrm{min})$ . Between runs, the capillary was rinsed with MeOH (3 min) and BGE (6 min). All condition- ing steps were performed at 2 bar.

The optimum BGE was found to consist of  $2\,\mathrm{M}$  formic acid and  $70\,\mathrm{mM}$  ammonium acetate in 60:40 acetoni- trile:methanol. The capillary cartridge was kept at  $20^{\circ}C$  The actual capillary temperature may have varied by a few degrees depending on the room temperature level by effect of the CE equipment being coupled to the mass spectro- meter. As recommended, the sheath liquid consisted of an  $80{:}20\,\mathrm{v/v}$  methanol–water mixture containing  $2\%$  formic acid and was used at a ﬂow rate of  $4\,\upmu\mathrm{L}/\mathrm{min}$ .

Samples were kept at ambient temperature in the autosampler and injected in the hydrodynamic mode at 50 mbar for 5 s, followed by running buffer, also at 50 mbar, for 3 s. Tests were carried out in the positive polarity mode, using a constant voltage of  $25\,\mathrm{kV}$ .

The ESI and MS operating conditions were as follows: fragmenter voltage,  $70\,\mathrm{V}$ ; nebulizer pressure, 1 psi;  $\mathrm{N}_{2}$  (drying gas) ﬂow rate, 3 L/min;  $\Nu_{2}$  (drying gas) temperature,  $150^{\circ}\mathrm{C}_{\mathrm{i}}$  monitoring the ions with the  $m/z$  values corresponding to . Such values were 235.2 for lidocaine, 247.2 for ${[\mathrm{{M}}+\mathrm{{H}}]}^{+}$  mepivacaine and 289.2 for bupivacaine. Quantiﬁcation was based on the corresponding ion for each individual analyte. The spray chamber was cleaned with methanol and Milli- $\mathsf Q$  water on a weekly basis. Also, daily cleaning with Milli- $\mathsf Q$  water is recommended.

### 2.4 Automatic setup for sample preconcentration

Sample treatment was performed with a  $200\,\upmu\mathrm{L}$  syringe (SGE Analytical Science) ﬁtted to a microextraction by packed sorbents system (MEPS) (SGE Analytical Science); each barrel insert and needle contained  ${\sim}4\,\mathrm{mg}$  of C18 packing of particle size  $45\,\upmu\mathrm{m}$  and pore size  $60\,\mathrm{\AA}$ .

The system was automated by connecting the syrin- ge–MEPS assembly to a syringe pump (Cavro) that was controlled via a computer running the software Sagittarius 3.0, which was obtained from the working group on Chemical Analysis and Vibrational Spectroscopy of the Vienna University of Technology, Austria.

For the analysis of free drugs, a microdialysis probe was on-line connected to the needle of the syringe-MEPS system. The microdialysis probe was a CMA 20 Elite Microdialysis model from CMA/Microdialysis AB (Stockholm, Sweden) withthefollowingspeciﬁcations:membrane,poly- arylethersulfone; membrane length,  $10\,\mathrm{mm}_{\mathrm{i}}$  membrane od,  $0.5\,\mathrm{mm}_{\mathrm{i}}$ ; cutoff,  $20\,000\,\mathrm{Da}$ ; shaft length,  $14\:\mathrm{mm}$ .

The MEPS system was conditioned with methanol ﬁrst and then water prior to ﬁrst use. In addition, it was washed with  methanol after each run in order to minimize carryover. $100\,\upmu\mathrm{L}$ 

### 2.5 Sample preparation

Human plasma and serum samples were centrifuged at  $4500\,\mathrm{rpm}$  for 5 min. The sample preparation procedure used to determine free concentrations of the target analytes in human plasma or serum involved adding  $40\,\mathrm{g/L}$  NaCl to  of sample and adjustment to pH 7.4. On the other $200\,\upmu\mathrm{L}$  hand, determinations of total analyte concentrations required that samples be treated as follows: mixtures containing 250 or  of sample and 50 or  $100\,\upmu\mathrm{L}$  $500\,\upmu\mathrm{L}$  of trichloroacetic acid were centrifuged at  $5000\,\mathrm{rpm}$  for 15 min [39], the supernatants being collected and adjusted to  $\mathrm{pH}~7.4\$  for analysis.

## 3Results and discussion

The structures of target analytes, three local anesthetics, are shown in Fig. 1. The results obtained in the determination of their free and total concentrations in human plasma are presented and discussed below.

### 3.1 NACE separation and MS detection

Based on the compatibility of the BGEs involved with MS detection, we chose to use NACE on the grounds that it avoids the use of a BGE with high capillary current intensity. Initially, we used an MeOH-based BGE. However, due to the strong EOF resulting from the low viscosity and high dielectric constant of the alcohol the addition of acetonitrile was required in order to improve selectivity and to reduce analysis times. Acetonitrile contents in the mixture up to  $60\%$  resulted in markedly improved baseline stability and resolution; on the other hand, contents above  $60\%$  raised the current intensity without increasing resolution. Therefore, a  $60{\cdot}40\,\mathrm{v}/\mathrm{v}$  acetoni- trile–methanol mixture was selected for further testing. We chose to use ammonium acetate at concentrations ranging from 35 to  $80\,\mathrm{mM}$  as electrolyte. The results thus obtained suggested that a  $70\,\mathrm{mM}$  concentration sufﬁced to ensure a stable current intensity and good separation. The effect adding formic acid to the BGE was also examined. Using an acid medium would cause the analytes to be protonated and facilitate their separa- tion as cations. Improving separation to a substantial extent required adding a 2 M concentration of formic acid, which had little effect on the current intensity. Injection was performed in the hydrodynamic mode by applying 50 mbar for 5 s, followed by BGE, also at 50 mbar, for 3 s. Increasing the sample injection time beyond the recommended value resulted in current interruption, so it was discarded. The effect of the applied voltage on the separation was examined over the range  $15{-}25\,\mathrm{kV}.$  Neither was any Joule effect observed nor were any exceedingly high current intensity values obtained over such a range, so a voltage of  $25\,\mathrm{kV}$  was adopted.



Spray chamber temperatures from  $120–200^{\circ}\mathsf{C}$  were studied.Thesensitivitywasfoundtodecreasewith increasing chamber temperature. On the other hand, reso- lution improved as the temperature was raised. A compro- mise must therefore be made between these two analytical properties. A temperature of  $150^{\circ}C$ resulted in only slightly reduced sensitivity, but markedly improved resolution, so it was chosen for further testing. The sheath liquid composi- tion is known to affect both electrospray stability and the ionization process. We studied various mixtures of metha- nol, acetonitrile, water and formic acid. Based on the results, an  $80{:}20\,\mathrm{v/v}$ methanol–water mixture containing  $2\%$  formic acid provided the best results in terms of electric contact and sensitivity. Sheath liquid ﬂow rates ranging from 2 to  $6\,\upmu\mathrm{L}/\upmu$  min were studied. The highest ﬂow rate resulted in lower sensitivity than the lowest one. A  $4\,\upmu\mathrm{L}/\mathrm{min}$  value was adopted as optimal since it provided better sensitivity than the lowest rate.

The nebulizer pressure has a critical inﬂuence on resolution. Pressure values from 1 to 6 psi were studied here. Pressures above 4 psi were found to result in current interruptions. Resolution was better at the lower pressures by effect of a reduced Venturi effect of the nebulizing gas at the capillary tip. Therefore, a pressure of 1 psi was selected as optimal. We also examined the inﬂuence of the ﬂow rate of drying gas, which was studied over the range  $1{-}6\;\mathrm{L}/\mathrm{min};$  based on the results, the best sensitivity was achieved with  $3\;\mathrm{L/min}$ . Finally, the inﬂuence of the fragmenter voltage on the ionization process was studied over the range 60–120 V and a value of  $70\,\mathrm{V}$  was selected.

### 3.2 MEPS preconcentration

The free fraction of the proposed analytes in human plasma was analyzed by using the microdialysis probe as coupled to MEPS. The microdialysis probe allows the easy separation of the free analytes from the analytes bounded to proteins that cannot pass through the microdialysis membrane. Preli- minary MEPS tests were performed on water samples spiked with a  $100\,\upmu\mathrm{g/L}$  concentration of each standard; then, an aliquot of  $200\,\upmu\mathrm{L}$  was preconcentrated at  $10\,\upmu\mathrm{L}/\mathrm{s}$ . Before  at $(2\times200\,\upmu\mathrm{L}$   and then eluted with methanol four times $100\,\upmu\mathrm{L}/\mathrm{s})$  . The analysis of the four consecutive extractions $(20\,\upmu\mathrm{L}\times4)$  revealed the presence of traces of the analytes in the second and third eluates, but none in the fourth. Based on these results, we reduced the analyte concentrations down to . However, the consecutive eluates still contained the $5\,\upmu\mathrm{g/L}$  analytes in proportions of  $29.3{-}42.3~\%$  with respect to the ﬁrst. This led us to add formic acid to the eluent in order to facilitate quantitative elution. The formic acid altered the distribution of the analytes in the sorbent and eluent. The acid concentrations studied ranged from 5 to  $200\,\mathrm{mM}$  (see Fig. 2). With  $10\,\mathrm{mM}$  formic acid, the second eluate exhibited  $9.8–20\%$  carryover. Based on the results, using  $20\,\mathrm{mM}$  concentration sufﬁced to ensure quantitative elution of the analytes since carryover was reproducible and always less than  $6\%$ .



Then, the effect of  $\mathrm{pH}$  was examined over the range 3.6–9.5, where no signiﬁcant changes in sensitivity were observed. Therefore,  $\mathrm{pH}~7.4$  was chosen in order to avoid differences with plasma  $\mathrm{pH}$ .

The RSD obtained for six spiked water samples under these conditions was high. This was ascribed to errors arising during the analytical procedure (particularly from human manipulation, the low volume of eluent used and capillary conditioning). In order to improve repeatability, we increased the conditioning time from 3 to  $6\,\mathrm{min}$ , which resulted in dramatically reduced RSD values. Also, increas- ing the eluent volume from 20 to  $50\,\upmu\mathrm{L}$  resulted in further improved RSD. Finally, using a syringe pump substantially reduced the need for human intervention and its associated errors as a result. In addition, incorporation of the syringe pump enabled the use of small eluent volumes (down to ) with good RSD values. $25\,\upmu\mathrm{L}$  The above-described procedure was used to examine the effect of coupling the microdialysis probe via a small piece  $(0.5\,\mathrm{cm})$  of Teﬂon tubing. Figure 3 illustrates the coupling of the automated MEPS system by using a syringe pump withacoupledmicrodialysisprobe.Theprobewas immersed in the sample, held in a vial furnished with a glass insert. As expected, preliminary tests showed that the ﬁber was permeable to the target analytes. However, using the above-described procedure unaltered resulted in strongly diminishedsensitivity.Inmicrodialysis,theabsolute recovery (mol/time unit) of a compound from the sample depends on (i) the ‘‘cut-off’’ of the dialysis membrane, (ii) the length of the membrane, (iii) the ﬂow rate of the and  $10\;S_{\mathrm{y/x}}/$ slope, respectively. The former ranged from 10.4 to  $15.2\,\upmu\mathrm{g/L}$  for free concentrations and 0.6 to  $1.6\,\upmu\mathrm{g/L}$  for total concentrations. The RSD values for peak areas and migration times were under 9 and  $1.4\%,$ , respectively, for both methods. As can be seen from the table, preconcentra- tion took  $35\,\mathrm{min}$  with MEPS-microdialysis and only  $2\,\mathrm{min}$  with the MEPS procedure. LODs were markedly better with MEPS than with microdialysis and MEPS in combination (viz. 0.6 versus  $10.4\,\upmu\mathrm{g/L})$ . This testiﬁes to the better suitability of MEPS than microdialysis–MEPS for the determination of total analyte concentrations.



perfusion ﬂuid and (iv) the diffusion coefﬁcient of the compound through the ﬂuid. Because the former two variables had previously been examined and adjusted, we focused on the latter two. First, we examined the effect of the ﬂow rate of perfusion ﬂuid (Milli-Q water) and hence the contact time between the sample and perfusion ﬂuid, and found increasing the contact time between the sample and ﬁber to boost diffusion of the analytes through the perfusion ﬂuid. The contact time could be improved by holding the ﬂow sequentially or by reducing the ﬂow rate. We performed several tests in order to study the effect of the contact time between sample and ﬁber. The best diffusion rates were obtained with sequential loadings of  $5\,\upmu\mathrm{L}$ at  $0.1\,\upmu\mathrm{L}/\mathrm{s}\!+\!60\,\mathrm{s}$  waiting time 40 times. The total sample volume passed through the sorbent was thus  $100\,\upmu\mathrm{L}.$  However, a long time was required that was successfully shortened by using a continuous sequence without breaks at 0.1 mL/s and then repeating the process another time (100 mL 2).

The next step involved modifying the ionic strength of the sample media. For this purpose, NaCl at concentrations ranging from 10 to 200 g/L was added to the samples. The best recov- eries were obtained with 40 g/L. On the other hand, concen- trations above 200 g/L resulted in ﬁber caking and clogging.

The sample treatment used for the determination of total analyte concentrations in human plasma is described in Section 2. The supernatant collected from the plasma follow- ing treatment with trichloroacetic acid and centrifugation was adjusted to pH 7.4 and preconcentrated with MEPS at 2 mL/s, which was slow enough to ensure optimum analyte retention in the C18 bed. Elution was done with 25 mL of eluent. As stated above, this volume was large enough to ensure repetitive results (RSD ranged from 4.5 to 7.6%).

### 3.4 Analysis of human plasma

a) In these samples, the loaded volume was reduced down to . $250\,\upmu\mathsf{L}$  To demonstrate this interaction with proteins, several tests were done on human serum as sample because of their low content in proteins. As can be seen in Table 2, the recoveries for lidocaine, mepivacaine and bupivacaine spanned the ranges 34.4–37.7, 44.7–46 and  $11.7{-}12.7\%$ , respectively. As expected, the recoveries from serum samples exceeded those from plasma samples by effect of the former containing less protein and a larger free fraction of analytes as a result.



Table 3 shows the ﬁgures of merit of the determination of total analyte concentrations in different plasma samples. As can be seen, samples 1, 6 and 7 tested positive for lido- caine. In addition, analytes recoveries were determined in four different samples spiked with a  $10\,\upmu\mathrm{g/L}$  concentration of each analyte. The estimated recoveries ranged from 78 to  $92\%$  for lidocaine, 58.39 to  $68.7\%$  for mepivacaine and 42.5 to  $56.59\%$  for bupivacaine. As expected, drug recoveries were higher than with the free concentration method by effect of bonds with proteins being broken and the resulting free drugs joining the free fraction. We should note that a volume of sample as low as  $250\,\upmu\mathrm{L}$  should sufﬁce for the intended purpose since samples 6 and 7 exhibited no signiﬁcant difference from sample 1, which was analyzed by using  $500\,\upmu\mathrm{L}$ .



The electropherograms of Fig. 4 expose the differences between the free concentration method and the total concentration method. Figure 4A corresponds to an aliquot of a single plasma sample analyzed with the free concen- tration method, both unspiked (1) and spiked with  $1.6\,\mathrm{mg/L}$  lidocaine, and  $1.4\,\mathrm{mg/L}$  mepivacaine and bupivacaine (2).

Figure 4B shows the electropherograms obtained in the total concentration analysis of other aliquots of the same plasma sample, both unspiked (1) and spiked with  $5\,\upmu\mathrm{g/L}$  lidocaine, and  $4.4\,\upmu\mathrm{g/L}$  mepivacaine and bupivacaine (2).

Figure5showstwoselectedelectropherograms obtained by using the total concentration method. Finally, Fig. 5 (1) corresponds to an unspiked human plasma sample aliquot, which tested positive for lidocaine, and Fig. 5 (2) to an aliquot of the same sample spiked with up to  lidocaine. $10\,\upmu\mathrm{g/L}$ 

## 4Concluding remarks

The high potential of the combination MEPS-CE has been demonstrated. MEPS provides the efﬁcient preconcentration of analytes from a small sample volume. In addition, its automatization has been achieved by using a simple syringe pump. The use of a microdialysis needle on-line coupled to the MEPS-syringe had permitted the analysis of free and total drug in human plasma. This simple strategy is then recommended to perform biological studies.

The authors are grateful to Spain’s Ministry of Education and Science for funding this work as a part of Project 2007- 60426. G.M.-C. also wishes to thank the Ministry for the award of a Research Training Fellowship (Grant BES-2005-10603).

The authors have declared no conﬂict of interest.

## 5References

[1] Drewe, J., Rufer, S., Huwyler, J., Ku¨ sters, E., matogr. B 1997, 691, 105–110

[2] Abdel-Rehim, M., Bielenstein, M., Askemark, Y., Tyre- fors, N., Arvidsson, T., J. Chromatogr. B 2000, 741, 175–188.

[3] Kang, L ., Jun, H. W., McCall, J. W., J. Pharm. Biomed. Anal. 1999, 19, 737–745.

[4] Reif, S., Le Corre, P., Dollo, G., Chevanne, F., Le Verge, R., J. Chromatogr. B 1998, 719, 239–244.

[5] Arvidsson, T., Eklund, E., J. Chromatogr. B 1995, 668, 91–98.

[6] Arvidsson, T., Askemark, Y., Halldin, M. M., Biomed. Chromatogr. 1999, 13, 286–292.

[7] Rifai, N., Hsin, O., Hope, T., Sakamoto, M., Ther. Drug Monit. 2001, 23, 182–186.

[8] Altun, Z., Abdel-Rehim, M., Blomberg, L. G., J. Chro- matogr. B 2004, 813, 129–135.

[9] Abdel-Rehim, M., Andersson, M., Portelius, E., Norsten- Ho¨ o¨ g, C., Blomberg, L.G.,, 313–321.

[10] Koster, E. H. M., Wemes, C., Morsink, J. B., de Jong, G. J., J. Chromatogr. B 2000, 739, 175–182.

[11] Abdel-Rehim,M.,Bielenstein,M.,Arvidsson,T., J. Microcol. Sep. 2000, 12, 308–315.

[12] Wynne, P., Hibbert, R., DiFeo, D., Dawes, P., The Column. 2008, 4, 12– 17.

[13] El-Beqqali, A., Abdel-Rehim, M., J. Sep. Sci. 2007, 30, 2501–2505.

[14] Said, R., Hassan, Z., Hassan, M., Abdel-Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2008, 31, 683–694.

[15] Abdel-Rehim, M., Altun, Z., Blomberg, L., J. Mass Spectrom. 2004, 39, 1488–1493.

[16] Abdel-Rehim, M., Hassan, Z., Skansem, P., Hassan, M., J. Liq. Chromatogr. Rel. Technol. 30, 3029–3041.

[17] Abdel-Rehim, M., Skansen, P., Vita, M., Hassan, Z., Blomberg, L., Hassan, M., Anal. Chim. Acta 2005, 539, 35–39.

[18] Vita, M., Skansen, P., Hassan, M., Abdel-Rehim, M., J. Chromatogr. B 2005, 817, 303–307.

[19] Abdel-Rehim, M ., Andersson, L. I., Altun, Z., Blomberg, L. G., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 1725–1736.

[20] El-Beqqali,A.,Kussak,A.,Blomberg,L.,Abdel- Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2007, 30, 575–586.

[21] Abdel-Rehim, M., Dahlgren, M., Blomberg, L., Claude, S., Tabacchi, R., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 2537–2544.

[22] Abdel-Rehim, M ., J. Chromatogr. B 2004, 801, 317–321.

[23] Al-Nouti, Y., Bartlett, M. G., J. Am. Soc. Mass Spectrom. 2002, 13, 928–935.

[24] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[25] Chee, G. L., Wan, T. S. M., J. Chromatogr. B 1993, 612, 172–177.

[26] Wang, Z., Wan, H., Anderson, M. S., Abdel-Rehim, M., Blomberg, L. G., Electrophoresis 2001, 22, 2495–2502.

[27] Geiser, L., Rudaz, S., Veuthey, J. L., Electrophoresis 2003, 24, 3049–3056.

[28] Curcuruto, O., Zaramella, A., Hamdan, M., Turrina, S., Tagliaro, F., Rapid Commun. Mass Spectrom. 1995, 9, 1487–1491.

[29] Lurie, I. S., Hays, P. A., Parker, K., Electrophoresis 2004, 25, 1580–1591.

[30] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[31] Anderson, M. S., Wan, H., Abdel-Rehim, M., Blomberg, L. G., J. Microcol. Sep. 1999, 11, 620–626.

[32] Bretnall, A. E., Clarke, G. S., J. Chromatogr. A 1996, 745, 145–154.

[33] Bjørnsdottir, I ., Hansen, S. H., J. Biochem. Biophys. Methods 1999, 38, 155–161.

[34] Bjørnsdottir, I., Tjørrnelund, J., Hansen, S. H., Electro- phoresis 1998, 19, 2179–2186.

[35] Cherkaoui, S., Geiser, L., Veuthey, J. L., Chromato- graphia 2000, 52, 403–407.

[36] Zhang, J., Lou, Y., Hoogmartens, J., Van Schepdael, A., Electrophoresis 2006, 27, 4827–4835.

[37] Cherkaoui,S.,Veuthey,J.L.,Analysis1999,27, 765–771.

[38] Anderson, M. S., Lu, B., Abdel-Rehim, M., Blomberg, S., Blomberg, L. G., Rapid Commun. Mass Spectrom. 2004, 18, 2612–2618.

[39] March, J., Simonet, B. M., Grases, F., J. Chromatogr. B 2001, 757, 247–255.
Gabriel Morales-Cid Soledad Ca´ rdenas Bartolome´ M. Simonet Miguel Valca´ rcel Department of Analytical Chemistry, Campus de Rabanales, University of Cordoba, Cordoba, Spain Received July 29, 2008 Revised October 23, 2008 Accepted October 26, 2008

 



# Research Article

# Direct automatic determination of free and total anesthetic drugs in human plasma by use of a dual (microdialysis–microextraction by packed sorbent) sample treatment coupled at-line to NACE–MS

This paper reports for the ﬁrst time the use of microextraction by packed sorbent in combination with CE. The combined system was used to determine anesthetic drugs in human plasma. A microdialysis ﬁber was coupled on-line to the microextraction unit in order to distinguish between free and total concentrations of drugs. The system was automated by connecting the microextraction unit to a syringe pump and interfacing it to a computer. The ensuing method allows the determination of  $10\,\upmu\mathrm{g/L}$  concentrations of free drugs and  $1\,\upmu\mathrm{g/L}$  concentrations of total drugs from only  $200\,\upmu\mathrm{L}$  of sample with an RSD of less than  $9\%$ .

### Keywords: Anesthetics  $/$  Human plasma / Microdialysis / Microextraction by packed sorbent / NACE-MSDOI 10.1002/elps.200800493

## 1Introduction

Obtaining quality analytical information in pharmacology entails minimizing sample preparation and analysis times. Sample treatments are especially important in this ﬁeld owing to the high complexity of some sample matrices such as plasma and urine. Sample preparation should be simpliﬁed by using automated devices and expeditious, environmentally friendly, inexpensive methodologies invol- ving reduced human intervention. In addition, procedures shouldbehighlyreproducible,providehighanalyte recoveriesandfacilitateenrichmentwiththetarget compounds.

This work was undertaken with the aim of simplifying sample treatment for the determination of local anesthetics such as lidocaine, mepivacaine and bupivacaine in human plasma, which should be administered in the minimal dose required by each individual patient according to age, weight and health status.

Solid-phase extraction (SPE) is probably the most widely used sample preparation technique by virtue of its affording concentration and cleanup of target compounds from a complex sample matrix. Among other applications, SPE has been used in combination with HPLC for the extraction of local anesthetics in human plasma [1–7]. Although it usually provides good recoveries  $(\sim\!70\%)$  of sorbent (more than  $100\,\mathrm{mg})$  and also large sample volumes  $(\geq1\,\mathrm{mL})$  been used for the extraction of local anesthetics in plasma samples [9–11]; this technique, however, uses large sample volumes  $(\geq1\,\mathrm{mL})$  , provides low recoveries and has the additional $(>40\,\mathrm{min})$  disadvantage of the short useful life of the ﬁbers.



Microextrationbypackedsorbent(MEPS),which is also known as ‘‘microextraction by packed syringe’’, integrates a miniaturized SPE bed with a microvolume syringe( $.100{-}250\,\upmu\mathrm{L})$ .CommercialMEPSequipment uses an integrated barrel insert and needle device to reduce SPE to a microscale suitable for small volume samples and for the on-line adaptation of conventional SPEtechniques.Inaddition,MEPSequipmentis portable and can be operated manually or connected to a robotic sampler [12]. MEPS features more expeditious sample preparation and uses organic solvents more spar- ingly than both liquid–liquid extraction and SPE [13]. In fact, MEPS minimizes sample volumes, reduces sample preparation times and solvent volumes, and combines sample extraction and sample injection, which is done directly in LC and GC. Among others, MEPS has been used in combination with LC–MS [8, 14–16], LC-MS/MS [17–21] and GC-MS [8, 15, 21, 22] for the determination of drugs in However, MEPS has never previously been used in combination with CE. Although the CE technique features a high resolution power, it has limited sensitivity, even when coupled to an MS detector. This requires using some sample concentration pretreatment in order to ensure acceptable analytical performance. The small sample volumes used in CE (a few nanoliters only) can be highly advantageous with a view to maximizing analyte enrichment by using a well- designed sample treatment. Therefore, combining MEPS and CE can provide interesting advantages. Local anesthetics and their metabolites have so far been determined by CZE [23–28], EKC [12, 29–32] and NACE [26, 33–38].



The purpose of this work was to assess the potential of the at-line coupling of MEPS to CE–MS for the determina- tion of free and total concentrations of local anesthetics in human plasma. The coupling of the MEPS-CE/MS combi- nation in-line to microdialysis ﬁber is also described in the paper to perform the rapid and automatic analysis of free drugs.

## 2Materials and methods

### 2.1 Reagents and standards

Lidocaine,mepivacainehydrochlorideandbupivacaine hydrochloride (Sigma–Aldrich; Madrid, Spain) were used as standards. Ammonium acetate, formic acid (Sigma– Aldrich;), methanol and acetonitrile (Panreac Quı´mica, S. A.; Barcelona, Spain) were used to prepare the BGE. Sodium hydroxide and hydrochloric acid (Panreac Quı´mica) were used for  $\mathrm{pH}$  adjustment and capillary conditioning. Finally, trichloroacetic acid and sodium chloride (Panreac Quı´mica) were used for sample treatment.

Standard stock solutions of the target analytes were prepared by dissolving in methanol or water the amount requiredtoobtaina $50\,\mathrm{mg/L}$ concentrationofeach compound and stored at  $4^{\circ}C$  until use. An acid  $\mathrm{pH}$  was required to dissolve the analytes in water, which was thus supplied with  $1\%$  formic acid for this purpose. Standard solutions of appropriate concentrations were prepared on a daily basis by diluting the stock solutions in methanol.

### 2.2 Instrumentation

An Agilent  $\mathrm{HP}^{3\mathrm{D}}$  Capillary Electrophoresis system (Wald- bronn, Germany) coupled to an Agilent 1100 Series LC/ MSD mass spectrometer via an electrospray-API interface was used. The make-up ﬂow of sheath liquid was delivered by the Agilent 1100 isocratic pump, which was operated at a 1:100 split ratio. The uncoated fused-silica capillaries used  $(80\,\mathrm{cm}\times50\,\upmu\mathrm{m}$  $375\,\upmu\mathrm{m}$  od) were supplied by Beckman (Fullerton, CA). The CE-MS system was controlled, and data wereacquiredandprocessed,byusingtheAgilent ChemStation software.

### 2.3 Instrumental operating conditions

The BGE employed was previously used elsewhere [38]. Each new capillary was initially conditioned with  $1\,\mathrm{M\,\,HCl}$  (5 min), 0.1 M NaOH (10 min) and Milli- $\mathsf Q$  water (5 min), in this sequence. Each day, the capillary in use was conditioned with 0.1 M NaOH ( $(10\,\mathrm{min})$ , Milli-Q water  $(5\,\mathrm{min})$  and running buffer  $(5\,\mathrm{min})$ . Between runs, the capillary was rinsed with MeOH (3 min) and BGE (6 min). All condition- ing steps were performed at 2 bar.

The optimum BGE was found to consist of  $2\,\mathrm{M}$  formic acid and  $70\,\mathrm{mM}$  ammonium acetate in 60:40 acetoni- trile:methanol. The capillary cartridge was kept at  $20^{\circ}C$  The actual capillary temperature may have varied by a few degrees depending on the room temperature level by effect of the CE equipment being coupled to the mass spectro- meter. As recommended, the sheath liquid consisted of an  $80{:}20\,\mathrm{v/v}$  methanol–water mixture containing  $2\%$  formic acid and was used at a ﬂow rate of  $4\,\upmu\mathrm{L}/\mathrm{min}$ .

Samples were kept at ambient temperature in the autosampler and injected in the hydrodynamic mode at 50 mbar for 5 s, followed by running buffer, also at 50 mbar, for 3 s. Tests were carried out in the positive polarity mode, using a constant voltage of  $25\,\mathrm{kV}$ .

The ESI and MS operating conditions were as follows: fragmenter voltage,  $70\,\mathrm{V}$ ; nebulizer pressure, 1 psi;  $\mathrm{N}_{2}$  (drying gas) ﬂow rate, 3 L/min;  $\Nu_{2}$  (drying gas) temperature,  $150^{\circ}\mathrm{C}_{\mathrm{i}}$  monitoring the ions with the  $m/z$  values corresponding to . Such values were 235.2 for lidocaine, 247.2 for ${[\mathrm{{M}}+\mathrm{{H}}]}^{+}$  mepivacaine and 289.2 for bupivacaine. Quantiﬁcation was based on the corresponding ion for each individual analyte. The spray chamber was cleaned with methanol and Milli- $\mathsf Q$  water on a weekly basis. Also, daily cleaning with Milli- $\mathsf Q$  water is recommended.

### 2.4 Automatic setup for sample preconcentration

Sample treatment was performed with a  $200\,\upmu\mathrm{L}$  syringe (SGE Analytical Science) ﬁtted to a microextraction by packed sorbents system (MEPS) (SGE Analytical Science); each barrel insert and needle contained  ${\sim}4\,\mathrm{mg}$  of C18 packing of particle size  $45\,\upmu\mathrm{m}$  and pore size  $60\,\mathrm{\AA}$ .

The system was automated by connecting the syrin- ge–MEPS assembly to a syringe pump (Cavro) that was controlled via a computer running the software Sagittarius 3.0, which was obtained from the working group on Chemical Analysis and Vibrational Spectroscopy of the Vienna University of Technology, Austria.

For the analysis of free drugs, a microdialysis probe was on-line connected to the needle of the syringe-MEPS system. The microdialysis probe was a CMA 20 Elite Microdialysis model from CMA/Microdialysis AB (Stockholm, Sweden) withthefollowingspeciﬁcations:membrane,poly- arylethersulfone; membrane length,  $10\,\mathrm{mm}_{\mathrm{i}}$  membrane od,  $0.5\,\mathrm{mm}_{\mathrm{i}}$ ; cutoff,  $20\,000\,\mathrm{Da}$ ; shaft length,  $14\:\mathrm{mm}$ .

The MEPS system was conditioned with methanol ﬁrst and then water prior to ﬁrst use. In addition, it was washed with  methanol after each run in order to minimize carryover. $100\,\upmu\mathrm{L}$ 

### 2.5 Sample preparation

Human plasma and serum samples were centrifuged at  $4500\,\mathrm{rpm}$  for 5 min. The sample preparation procedure used to determine free concentrations of the target analytes in human plasma or serum involved adding  $40\,\mathrm{g/L}$  NaCl to  of sample and adjustment to pH 7.4. On the other $200\,\upmu\mathrm{L}$  hand, determinations of total analyte concentrations required that samples be treated as follows: mixtures containing 250 or  of sample and 50 or  $100\,\upmu\mathrm{L}$  $500\,\upmu\mathrm{L}$  of trichloroacetic acid were centrifuged at  $5000\,\mathrm{rpm}$  for 15 min [39], the supernatants being collected and adjusted to  $\mathrm{pH}~7.4\$  for analysis.

## 3Results and discussion

The structures of target analytes, three local anesthetics, are shown in Fig. 1. The results obtained in the determination of their free and total concentrations in human plasma are presented and discussed below.

### 3.1 NACE separation and MS detection

Based on the compatibility of the BGEs involved with MS detection, we chose to use NACE on the grounds that it avoids the use of a BGE with high capillary current intensity. Initially, we used an MeOH-based BGE. However, due to the strong EOF resulting from the low viscosity and high dielectric constant of the alcohol the addition of acetonitrile was required in order to improve selectivity and to reduce analysis times. Acetonitrile contents in the mixture up to  $60\%$  resulted in markedly improved baseline stability and resolution; on the other hand, contents above  $60\%$  raised the current intensity without increasing resolution. Therefore, a  $60{\cdot}40\,\mathrm{v}/\mathrm{v}$  acetoni- trile–methanol mixture was selected for further testing. We chose to use ammonium acetate at concentrations ranging from 35 to  $80\,\mathrm{mM}$  as electrolyte. The results thus obtained suggested that a  $70\,\mathrm{mM}$  concentration sufﬁced to ensure a stable current intensity and good separation. The effect adding formic acid to the BGE was also examined. Using an acid medium would cause the analytes to be protonated and facilitate their separa- tion as cations. Improving separation to a substantial extent required adding a 2 M concentration of formic acid, which had little effect on the current intensity. Injection was performed in the hydrodynamic mode by applying 50 mbar for 5 s, followed by BGE, also at 50 mbar, for 3 s. Increasing the sample injection time beyond the recommended value resulted in current interruption, so it was discarded. The effect of the applied voltage on the separation was examined over the range  $15{-}25\,\mathrm{kV}.$  Neither was any Joule effect observed nor were any exceedingly high current intensity values obtained over such a range, so a voltage of  $25\,\mathrm{kV}$  was adopted.



Spray chamber temperatures from  $120–200^{\circ}\mathsf{C}$  were studied.Thesensitivitywasfoundtodecreasewith increasing chamber temperature. On the other hand, reso- lution improved as the temperature was raised. A compro- mise must therefore be made between these two analytical properties. A temperature of  $150^{\circ}C$ resulted in only slightly reduced sensitivity, but markedly improved resolution, so it was chosen for further testing. The sheath liquid composi- tion is known to affect both electrospray stability and the ionization process. We studied various mixtures of metha- nol, acetonitrile, water and formic acid. Based on the results, an  $80{:}20\,\mathrm{v/v}$ methanol–water mixture containing  $2\%$  formic acid provided the best results in terms of electric contact and sensitivity. Sheath liquid ﬂow rates ranging from 2 to  $6\,\upmu\mathrm{L}/\upmu$  min were studied. The highest ﬂow rate resulted in lower sensitivity than the lowest one. A  $4\,\upmu\mathrm{L}/\mathrm{min}$  value was adopted as optimal since it provided better sensitivity than the lowest rate.

The nebulizer pressure has a critical inﬂuence on resolution. Pressure values from 1 to 6 psi were studied here. Pressures above 4 psi were found to result in current interruptions. Resolution was better at the lower pressures by effect of a reduced Venturi effect of the nebulizing gas at the capillary tip. Therefore, a pressure of 1 psi was selected as optimal. We also examined the inﬂuence of the ﬂow rate of drying gas, which was studied over the range  $1{-}6\;\mathrm{L}/\mathrm{min};$  based on the results, the best sensitivity was achieved with  $3\;\mathrm{L/min}$ . Finally, the inﬂuence of the fragmenter voltage on the ionization process was studied over the range 60–120 V and a value of  $70\,\mathrm{V}$  was selected.

### 3.2 MEPS preconcentration

The free fraction of the proposed analytes in human plasma was analyzed by using the microdialysis probe as coupled to MEPS. The microdialysis probe allows the easy separation of the free analytes from the analytes bounded to proteins that cannot pass through the microdialysis membrane. Preli- minary MEPS tests were performed on water samples spiked with a  $100\,\upmu\mathrm{g/L}$  concentration of each standard; then, an aliquot of  $200\,\upmu\mathrm{L}$  was preconcentrated at  $10\,\upmu\mathrm{L}/\mathrm{s}$ . Before  at $(2\times200\,\upmu\mathrm{L}$   and then eluted with methanol four times $100\,\upmu\mathrm{L}/\mathrm{s})$  . The analysis of the four consecutive extractions $(20\,\upmu\mathrm{L}\times4)$  revealed the presence of traces of the analytes in the second and third eluates, but none in the fourth. Based on these results, we reduced the analyte concentrations down to . However, the consecutive eluates still contained the $5\,\upmu\mathrm{g/L}$  analytes in proportions of  $29.3{-}42.3~\%$  with respect to the ﬁrst. This led us to add formic acid to the eluent in order to facilitate quantitative elution. The formic acid altered the distribution of the analytes in the sorbent and eluent. The acid concentrations studied ranged from 5 to  $200\,\mathrm{mM}$  (see Fig. 2). With  $10\,\mathrm{mM}$  formic acid, the second eluate exhibited  $9.8–20\%$  carryover. Based on the results, using  $20\,\mathrm{mM}$  concentration sufﬁced to ensure quantitative elution of the analytes since carryover was reproducible and always less than  $6\%$ .



Then, the effect of  $\mathrm{pH}$  was examined over the range 3.6–9.5, where no signiﬁcant changes in sensitivity were observed. Therefore,  $\mathrm{pH}~7.4$  was chosen in order to avoid differences with plasma  $\mathrm{pH}$ .

The RSD obtained for six spiked water samples under these conditions was high. This was ascribed to errors arising during the analytical procedure (particularly from human manipulation, the low volume of eluent used and capillary conditioning). In order to improve repeatability, we increased the conditioning time from 3 to  $6\,\mathrm{min}$ , which resulted in dramatically reduced RSD values. Also, increas- ing the eluent volume from 20 to  $50\,\upmu\mathrm{L}$  resulted in further improved RSD. Finally, using a syringe pump substantially reduced the need for human intervention and its associated errors as a result. In addition, incorporation of the syringe pump enabled the use of small eluent volumes (down to ) with good RSD values. $25\,\upmu\mathrm{L}$  The above-described procedure was used to examine the effect of coupling the microdialysis probe via a small piece  $(0.5\,\mathrm{cm})$  of Teﬂon tubing. Figure 3 illustrates the coupling of the automated MEPS system by using a syringe pump withacoupledmicrodialysisprobe.Theprobewas immersed in the sample, held in a vial furnished with a glass insert. As expected, preliminary tests showed that the ﬁber was permeable to the target analytes. However, using the above-described procedure unaltered resulted in strongly diminishedsensitivity.Inmicrodialysis,theabsolute recovery (mol/time unit) of a compound from the sample depends on (i) the ‘‘cut-off’’ of the dialysis membrane, (ii) the length of the membrane, (iii) the ﬂow rate of the and  $10\;S_{\mathrm{y/x}}/$ slope, respectively. The former ranged from 10.4 to  $15.2\,\upmu\mathrm{g/L}$  for free concentrations and 0.6 to  $1.6\,\upmu\mathrm{g/L}$  for total concentrations. The RSD values for peak areas and migration times were under 9 and  $1.4\%,$ , respectively, for both methods. As can be seen from the table, preconcentra- tion took  $35\,\mathrm{min}$  with MEPS-microdialysis and only  $2\,\mathrm{min}$  with the MEPS procedure. LODs were markedly better with MEPS than with microdialysis and MEPS in combination (viz. 0.6 versus  $10.4\,\upmu\mathrm{g/L})$ . This testiﬁes to the better suitability of MEPS than microdialysis–MEPS for the determination of total analyte concentrations.



perfusion ﬂuid and (iv) the diffusion coefﬁcient of the compound through the ﬂuid. Because the former two variables had previously been examined and adjusted, we focused on the latter two. First, we examined the effect of the ﬂow rate of perfusion ﬂuid (Milli-Q water) and hence the contact time between the sample and perfusion ﬂuid, and found increasing the contact time between the sample and ﬁber to boost diffusion of the analytes through the perfusion ﬂuid. The contact time could be improved by holding the ﬂow sequentially or by reducing the ﬂow rate. We performed several tests in order to study the effect of the contact time between sample and ﬁber. The best diffusion rates were obtained with sequential loadings of  $5\,\upmu\mathrm{L}$ at  $0.1\,\upmu\mathrm{L}/\mathrm{s}\!+\!60\,\mathrm{s}$  waiting time 40 times. The total sample volume passed through the sorbent was thus  $100\,\upmu\mathrm{L}.$  However, a long time was required that was successfully shortened by using a continuous sequence without breaks at 0.1 mL/s and then repeating the process another time (100 mL 2).

The next step involved modifying the ionic strength of the sample media. For this purpose, NaCl at concentrations ranging from 10 to 200 g/L was added to the samples. The best recov- eries were obtained with 40 g/L. On the other hand, concen- trations above 200 g/L resulted in ﬁber caking and clogging.

The sample treatment used for the determination of total analyte concentrations in human plasma is described in Section 2. The supernatant collected from the plasma follow- ing treatment with trichloroacetic acid and centrifugation was adjusted to pH 7.4 and preconcentrated with MEPS at 2 mL/s, which was slow enough to ensure optimum analyte retention in the C18 bed. Elution was done with 25 mL of eluent. As stated above, this volume was large enough to ensure repetitive results (RSD ranged from 4.5 to 7.6%).

### 3.4 Analysis of human plasma

a) In these samples, the loaded volume was reduced down to . $250\,\upmu\mathsf{L}$  To demonstrate this interaction with proteins, several tests were done on human serum as sample because of their low content in proteins. As can be seen in Table 2, the recoveries for lidocaine, mepivacaine and bupivacaine spanned the ranges 34.4–37.7, 44.7–46 and  $11.7{-}12.7\%$ , respectively. As expected, the recoveries from serum samples exceeded those from plasma samples by effect of the former containing less protein and a larger free fraction of analytes as a result.



Table 3 shows the ﬁgures of merit of the determination of total analyte concentrations in different plasma samples. As can be seen, samples 1, 6 and 7 tested positive for lido- caine. In addition, analytes recoveries were determined in four different samples spiked with a  $10\,\upmu\mathrm{g/L}$  concentration of each analyte. The estimated recoveries ranged from 78 to  $92\%$  for lidocaine, 58.39 to  $68.7\%$  for mepivacaine and 42.5 to  $56.59\%$  for bupivacaine. As expected, drug recoveries were higher than with the free concentration method by effect of bonds with proteins being broken and the resulting free drugs joining the free fraction. We should note that a volume of sample as low as  $250\,\upmu\mathrm{L}$  should sufﬁce for the intended purpose since samples 6 and 7 exhibited no signiﬁcant difference from sample 1, which was analyzed by using  $500\,\upmu\mathrm{L}$ .



The electropherograms of Fig. 4 expose the differences between the free concentration method and the total concentration method. Figure 4A corresponds to an aliquot of a single plasma sample analyzed with the free concen- tration method, both unspiked (1) and spiked with  $1.6\,\mathrm{mg/L}$  lidocaine, and  $1.4\,\mathrm{mg/L}$  mepivacaine and bupivacaine (2).

Figure 4B shows the electropherograms obtained in the total concentration analysis of other aliquots of the same plasma sample, both unspiked (1) and spiked with  $5\,\upmu\mathrm{g/L}$  lidocaine, and  $4.4\,\upmu\mathrm{g/L}$  mepivacaine and bupivacaine (2).

Figure5showstwoselectedelectropherograms obtained by using the total concentration method. Finally, Fig. 5 (1) corresponds to an unspiked human plasma sample aliquot, which tested positive for lidocaine, and Fig. 5 (2) to an aliquot of the same sample spiked with up to  lidocaine. $10\,\upmu\mathrm{g/L}$ 

## 4Concluding remarks

The high potential of the combination MEPS-CE has been demonstrated. MEPS provides the efﬁcient preconcentration of analytes from a small sample volume. In addition, its automatization has been achieved by using a simple syringe pump. The use of a microdialysis needle on-line coupled to the MEPS-syringe had permitted the analysis of free and total drug in human plasma. This simple strategy is then recommended to perform biological studies.

The authors are grateful to Spain’s Ministry of Education and Science for funding this work as a part of Project 2007- 60426. G.M.-C. also wishes to thank the Ministry for the award of a Research Training Fellowship (Grant BES-2005-10603).

The authors have declared no conﬂict of interest.

## 5References

[1] Drewe, J., Rufer, S., Huwyler, J., Ku¨ sters, E., matogr. B 1997, 691, 105–110

[2] Abdel-Rehim, M., Bielenstein, M., Askemark, Y., Tyre- fors, N., Arvidsson, T., J. Chromatogr. B 2000, 741, 175–188.

[3] Kang, L ., Jun, H. W., McCall, J. W., J. Pharm. Biomed. Anal. 1999, 19, 737–745.

[4] Reif, S., Le Corre, P., Dollo, G., Chevanne, F., Le Verge, R., J. Chromatogr. B 1998, 719, 239–244.

[5] Arvidsson, T., Eklund, E., J. Chromatogr. B 1995, 668, 91–98.

[6] Arvidsson, T., Askemark, Y., Halldin, M. M., Biomed. Chromatogr. 1999, 13, 286–292.

[7] Rifai, N., Hsin, O., Hope, T., Sakamoto, M., Ther. Drug Monit. 2001, 23, 182–186.

[8] Altun, Z., Abdel-Rehim, M., Blomberg, L. G., J. Chro- matogr. B 2004, 813, 129–135.

[9] Abdel-Rehim, M., Andersson, M., Portelius, E., Norsten- Ho¨ o¨ g, C., Blomberg, L.G.,, 313–321.

[10] Koster, E. H. M., Wemes, C., Morsink, J. B., de Jong, G. J., J. Chromatogr. B 2000, 739, 175–182.

[11] Abdel-Rehim,M.,Bielenstein,M.,Arvidsson,T., J. Microcol. Sep. 2000, 12, 308–315.

[12] Wynne, P., Hibbert, R., DiFeo, D., Dawes, P., The Column. 2008, 4, 12– 17.

[13] El-Beqqali, A., Abdel-Rehim, M., J. Sep. Sci. 2007, 30, 2501–2505.

[14] Said, R., Hassan, Z., Hassan, M., Abdel-Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2008, 31, 683–694.

[15] Abdel-Rehim, M., Altun, Z., Blomberg, L., J. Mass Spectrom. 2004, 39, 1488–1493.

[16] Abdel-Rehim, M., Hassan, Z., Skansem, P., Hassan, M., J. Liq. Chromatogr. Rel. Technol. 30, 3029–3041.

[17] Abdel-Rehim, M., Skansen, P., Vita, M., Hassan, Z., Blomberg, L., Hassan, M., Anal. Chim. Acta 2005, 539, 35–39.

[18] Vita, M., Skansen, P., Hassan, M., Abdel-Rehim, M., J. Chromatogr. B 2005, 817, 303–307.

[19] Abdel-Rehim, M ., Andersson, L. I., Altun, Z., Blomberg, L. G., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 1725–1736.

[20] El-Beqqali,A.,Kussak,A.,Blomberg,L.,Abdel- Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2007, 30, 575–586.

[21] Abdel-Rehim, M., Dahlgren, M., Blomberg, L., Claude, S., Tabacchi, R., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 2537–2544.

[22] Abdel-Rehim, M ., J. Chromatogr. B 2004, 801, 317–321.

[23] Al-Nouti, Y., Bartlett, M. G., J. Am. Soc. Mass Spectrom. 2002, 13, 928–935.

[24] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[25] Chee, G. L., Wan, T. S. M., J. Chromatogr. B 1993, 612, 172–177.

[26] Wang, Z., Wan, H., Anderson, M. S., Abdel-Rehim, M., Blomberg, L. G., Electrophoresis 2001, 22, 2495–2502.

[27] Geiser, L., Rudaz, S., Veuthey, J. L., Electrophoresis 2003, 24, 3049–3056.

[28] Curcuruto, O., Zaramella, A., Hamdan, M., Turrina, S., Tagliaro, F., Rapid Commun. Mass Spectrom. 1995, 9, 1487–1491.

[29] Lurie, I. S., Hays, P. A., Parker, K., Electrophoresis 2004, 25, 1580–1591.

[30] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[31] Anderson, M. S., Wan, H., Abdel-Rehim, M., Blomberg, L. G., J. Microcol. Sep. 1999, 11, 620–626.

[32] Bretnall, A. E., Clarke, G. S., J. Chromatogr. A 1996, 745, 145–154.

[33] Bjørnsdottir, I ., Hansen, S. H., J. Biochem. Biophys. Methods 1999, 38, 155–161.

[34] Bjørnsdottir, I., Tjørrnelund, J., Hansen, S. H., Electro- phoresis 1998, 19, 2179–2186.

[35] Cherkaoui, S., Geiser, L., Veuthey, J. L., Chromato- graphia 2000, 52, 403–407.

[36] Zhang, J., Lou, Y., Hoogmartens, J., Van Schepdael, A., Electrophoresis 2006, 27, 4827–4835.

[37] Cherkaoui,S.,Veuthey,J.L.,Analysis1999,27, 765–771.

[38] Anderson, M. S., Lu, B., Abdel-Rehim, M., Blomberg, S., Blomberg, L. G., Rapid Commun. Mass Spectrom. 2004, 18, 2612–2618.

[39] March, J., Simonet, B. M., Grases, F., J. Chromatogr. B 2001, 757, 247–255.
Gabriel Morales-Cid Soledad Ca´ rdenas Bartolome´ M. Simonet Miguel Valca´ rcel Department of Analytical Chemistry, Campus de Rabanales, University of Cordoba, Cordoba, Spain Received July 29, 2008 Revised October 23, 2008 Accepted October 26, 2008

 



# Research Article

# Direct automatic determination of free and total anesthetic drugs in human plasma by use of a dual (microdialysis–microextraction by packed sorbent) sample treatment coupled at-line to NACE–MS

This paper reports for the ﬁrst time the use of microextraction by packed sorbent in combination with CE. The combined system was used to determine anesthetic drugs in human plasma. A microdialysis ﬁber was coupled on-line to the microextraction unit in order to distinguish between free and total concentrations of drugs. The system was automated by connecting the microextraction unit to a syringe pump and interfacing it to a computer. The ensuing method allows the determination of  $10\,\upmu\mathrm{g/L}$  concentrations of free drugs and  $1\,\upmu\mathrm{g/L}$  concentrations of total drugs from only  $200\,\upmu\mathrm{L}$  of sample with an RSD of less than  $9\%$ .

### Keywords: Anesthetics  $/$  Human plasma / Microdialysis / Microextraction by packed sorbent / NACE-MSDOI 10.1002/elps.200800493

## 1Introduction

Obtaining quality analytical information in pharmacology entails minimizing sample preparation and analysis times. Sample treatments are especially important in this ﬁeld owing to the high complexity of some sample matrices such as plasma and urine. Sample preparation should be simpliﬁed by using automated devices and expeditious, environmentally friendly, inexpensive methodologies invol- ving reduced human intervention. In addition, procedures shouldbehighlyreproducible,providehighanalyte recoveriesandfacilitateenrichmentwiththetarget compounds.

This work was undertaken with the aim of simplifying sample treatment for the determination of local anesthetics such as lidocaine, mepivacaine and bupivacaine in human plasma, which should be administered in the minimal dose required by each individual patient according to age, weight and health status.

Solid-phase extraction (SPE) is probably the most widely used sample preparation technique by virtue of its affording concentration and cleanup of target compounds from a complex sample matrix. Among other applications, SPE has been used in combination with HPLC for the extraction of local anesthetics in human plasma [1–7]. Although it usually provides good recoveries  $(\sim\!70\%)$  of sorbent (more than  $100\,\mathrm{mg})$  and also large sample volumes  $(\geq1\,\mathrm{mL})$  been used for the extraction of local anesthetics in plasma samples [9–11]; this technique, however, uses large sample volumes  $(\geq1\,\mathrm{mL})$  , provides low recoveries and has the additional $(>40\,\mathrm{min})$  disadvantage of the short useful life of the ﬁbers.



Microextrationbypackedsorbent(MEPS),which is also known as ‘‘microextraction by packed syringe’’, integrates a miniaturized SPE bed with a microvolume syringe( $.100{-}250\,\upmu\mathrm{L})$ .CommercialMEPSequipment uses an integrated barrel insert and needle device to reduce SPE to a microscale suitable for small volume samples and for the on-line adaptation of conventional SPEtechniques.Inaddition,MEPSequipmentis portable and can be operated manually or connected to a robotic sampler [12]. MEPS features more expeditious sample preparation and uses organic solvents more spar- ingly than both liquid–liquid extraction and SPE [13]. In fact, MEPS minimizes sample volumes, reduces sample preparation times and solvent volumes, and combines sample extraction and sample injection, which is done directly in LC and GC. Among others, MEPS has been used in combination with LC–MS [8, 14–16], LC-MS/MS [17–21] and GC-MS [8, 15, 21, 22] for the determination of drugs in However, MEPS has never previously been used in combination with CE. Although the CE technique features a high resolution power, it has limited sensitivity, even when coupled to an MS detector. This requires using some sample concentration pretreatment in order to ensure acceptable analytical performance. The small sample volumes used in CE (a few nanoliters only) can be highly advantageous with a view to maximizing analyte enrichment by using a well- designed sample treatment. Therefore, combining MEPS and CE can provide interesting advantages. Local anesthetics and their metabolites have so far been determined by CZE [23–28], EKC [12, 29–32] and NACE [26, 33–38].



The purpose of this work was to assess the potential of the at-line coupling of MEPS to CE–MS for the determina- tion of free and total concentrations of local anesthetics in human plasma. The coupling of the MEPS-CE/MS combi- nation in-line to microdialysis ﬁber is also described in the paper to perform the rapid and automatic analysis of free drugs.

## 2Materials and methods

### 2.1 Reagents and standards

Lidocaine,mepivacainehydrochlorideandbupivacaine hydrochloride (Sigma–Aldrich; Madrid, Spain) were used as standards. Ammonium acetate, formic acid (Sigma– Aldrich;), methanol and acetonitrile (Panreac Quı´mica, S. A.; Barcelona, Spain) were used to prepare the BGE. Sodium hydroxide and hydrochloric acid (Panreac Quı´mica) were used for  $\mathrm{pH}$  adjustment and capillary conditioning. Finally, trichloroacetic acid and sodium chloride (Panreac Quı´mica) were used for sample treatment.

Standard stock solutions of the target analytes were prepared by dissolving in methanol or water the amount requiredtoobtaina $50\,\mathrm{mg/L}$ concentrationofeach compound and stored at  $4^{\circ}C$  until use. An acid  $\mathrm{pH}$  was required to dissolve the analytes in water, which was thus supplied with  $1\%$  formic acid for this purpose. Standard solutions of appropriate concentrations were prepared on a daily basis by diluting the stock solutions in methanol.

### 2.2 Instrumentation

An Agilent  $\mathrm{HP}^{3\mathrm{D}}$  Capillary Electrophoresis system (Wald- bronn, Germany) coupled to an Agilent 1100 Series LC/ MSD mass spectrometer via an electrospray-API interface was used. The make-up ﬂow of sheath liquid was delivered by the Agilent 1100 isocratic pump, which was operated at a 1:100 split ratio. The uncoated fused-silica capillaries used  $(80\,\mathrm{cm}\times50\,\upmu\mathrm{m}$  $375\,\upmu\mathrm{m}$  od) were supplied by Beckman (Fullerton, CA). The CE-MS system was controlled, and data wereacquiredandprocessed,byusingtheAgilent ChemStation software.

### 2.3 Instrumental operating conditions

The BGE employed was previously used elsewhere [38]. Each new capillary was initially conditioned with  $1\,\mathrm{M\,\,HCl}$  (5 min), 0.1 M NaOH (10 min) and Milli- $\mathsf Q$  water (5 min), in this sequence. Each day, the capillary in use was conditioned with 0.1 M NaOH ( $(10\,\mathrm{min})$ , Milli-Q water  $(5\,\mathrm{min})$  and running buffer  $(5\,\mathrm{min})$ . Between runs, the capillary was rinsed with MeOH (3 min) and BGE (6 min). All condition- ing steps were performed at 2 bar.

The optimum BGE was found to consist of  $2\,\mathrm{M}$  formic acid and  $70\,\mathrm{mM}$  ammonium acetate in 60:40 acetoni- trile:methanol. The capillary cartridge was kept at  $20^{\circ}C$  The actual capillary temperature may have varied by a few degrees depending on the room temperature level by effect of the CE equipment being coupled to the mass spectro- meter. As recommended, the sheath liquid consisted of an  $80{:}20\,\mathrm{v/v}$  methanol–water mixture containing  $2\%$  formic acid and was used at a ﬂow rate of  $4\,\upmu\mathrm{L}/\mathrm{min}$ .

Samples were kept at ambient temperature in the autosampler and injected in the hydrodynamic mode at 50 mbar for 5 s, followed by running buffer, also at 50 mbar, for 3 s. Tests were carried out in the positive polarity mode, using a constant voltage of  $25\,\mathrm{kV}$ .

The ESI and MS operating conditions were as follows: fragmenter voltage,  $70\,\mathrm{V}$ ; nebulizer pressure, 1 psi;  $\mathrm{N}_{2}$  (drying gas) ﬂow rate, 3 L/min;  $\Nu_{2}$  (drying gas) temperature,  $150^{\circ}\mathrm{C}_{\mathrm{i}}$  monitoring the ions with the  $m/z$  values corresponding to . Such values were 235.2 for lidocaine, 247.2 for ${[\mathrm{{M}}+\mathrm{{H}}]}^{+}$  mepivacaine and 289.2 for bupivacaine. Quantiﬁcation was based on the corresponding ion for each individual analyte. The spray chamber was cleaned with methanol and Milli- $\mathsf Q$  water on a weekly basis. Also, daily cleaning with Milli- $\mathsf Q$  water is recommended.

### 2.4 Automatic setup for sample preconcentration

Sample treatment was performed with a  $200\,\upmu\mathrm{L}$  syringe (SGE Analytical Science) ﬁtted to a microextraction by packed sorbents system (MEPS) (SGE Analytical Science); each barrel insert and needle contained  ${\sim}4\,\mathrm{mg}$  of C18 packing of particle size  $45\,\upmu\mathrm{m}$  and pore size  $60\,\mathrm{\AA}$ .

The system was automated by connecting the syrin- ge–MEPS assembly to a syringe pump (Cavro) that was controlled via a computer running the software Sagittarius 3.0, which was obtained from the working group on Chemical Analysis and Vibrational Spectroscopy of the Vienna University of Technology, Austria.

For the analysis of free drugs, a microdialysis probe was on-line connected to the needle of the syringe-MEPS system. The microdialysis probe was a CMA 20 Elite Microdialysis model from CMA/Microdialysis AB (Stockholm, Sweden) withthefollowingspeciﬁcations:membrane,poly- arylethersulfone; membrane length,  $10\,\mathrm{mm}_{\mathrm{i}}$  membrane od,  $0.5\,\mathrm{mm}_{\mathrm{i}}$ ; cutoff,  $20\,000\,\mathrm{Da}$ ; shaft length,  $14\:\mathrm{mm}$ .

The MEPS system was conditioned with methanol ﬁrst and then water prior to ﬁrst use. In addition, it was washed with  methanol after each run in order to minimize carryover. $100\,\upmu\mathrm{L}$ 

### 2.5 Sample preparation

Human plasma and serum samples were centrifuged at  $4500\,\mathrm{rpm}$  for 5 min. The sample preparation procedure used to determine free concentrations of the target analytes in human plasma or serum involved adding  $40\,\mathrm{g/L}$  NaCl to  of sample and adjustment to pH 7.4. On the other $200\,\upmu\mathrm{L}$  hand, determinations of total analyte concentrations required that samples be treated as follows: mixtures containing 250 or  of sample and 50 or  $100\,\upmu\mathrm{L}$  $500\,\upmu\mathrm{L}$  of trichloroacetic acid were centrifuged at  $5000\,\mathrm{rpm}$  for 15 min [39], the supernatants being collected and adjusted to  $\mathrm{pH}~7.4\$  for analysis.

## 3Results and discussion

The structures of target analytes, three local anesthetics, are shown in Fig. 1. The results obtained in the determination of their free and total concentrations in human plasma are presented and discussed below.

### 3.1 NACE separation and MS detection

Based on the compatibility of the BGEs involved with MS detection, we chose to use NACE on the grounds that it avoids the use of a BGE with high capillary current intensity. Initially, we used an MeOH-based BGE. However, due to the strong EOF resulting from the low viscosity and high dielectric constant of the alcohol the addition of acetonitrile was required in order to improve selectivity and to reduce analysis times. Acetonitrile contents in the mixture up to  $60\%$  resulted in markedly improved baseline stability and resolution; on the other hand, contents above  $60\%$  raised the current intensity without increasing resolution. Therefore, a  $60{\cdot}40\,\mathrm{v}/\mathrm{v}$  acetoni- trile–methanol mixture was selected for further testing. We chose to use ammonium acetate at concentrations ranging from 35 to  $80\,\mathrm{mM}$  as electrolyte. The results thus obtained suggested that a  $70\,\mathrm{mM}$  concentration sufﬁced to ensure a stable current intensity and good separation. The effect adding formic acid to the BGE was also examined. Using an acid medium would cause the analytes to be protonated and facilitate their separa- tion as cations. Improving separation to a substantial extent required adding a 2 M concentration of formic acid, which had little effect on the current intensity. Injection was performed in the hydrodynamic mode by applying 50 mbar for 5 s, followed by BGE, also at 50 mbar, for 3 s. Increasing the sample injection time beyond the recommended value resulted in current interruption, so it was discarded. The effect of the applied voltage on the separation was examined over the range  $15{-}25\,\mathrm{kV}.$  Neither was any Joule effect observed nor were any exceedingly high current intensity values obtained over such a range, so a voltage of  $25\,\mathrm{kV}$  was adopted.



Spray chamber temperatures from  $120–200^{\circ}\mathsf{C}$  were studied.Thesensitivitywasfoundtodecreasewith increasing chamber temperature. On the other hand, reso- lution improved as the temperature was raised. A compro- mise must therefore be made between these two analytical properties. A temperature of  $150^{\circ}C$ resulted in only slightly reduced sensitivity, but markedly improved resolution, so it was chosen for further testing. The sheath liquid composi- tion is known to affect both electrospray stability and the ionization process. We studied various mixtures of metha- nol, acetonitrile, water and formic acid. Based on the results, an  $80{:}20\,\mathrm{v/v}$ methanol–water mixture containing  $2\%$  formic acid provided the best results in terms of electric contact and sensitivity. Sheath liquid ﬂow rates ranging from 2 to  $6\,\upmu\mathrm{L}/\upmu$  min were studied. The highest ﬂow rate resulted in lower sensitivity than the lowest one. A  $4\,\upmu\mathrm{L}/\mathrm{min}$  value was adopted as optimal since it provided better sensitivity than the lowest rate.

The nebulizer pressure has a critical inﬂuence on resolution. Pressure values from 1 to 6 psi were studied here. Pressures above 4 psi were found to result in current interruptions. Resolution was better at the lower pressures by effect of a reduced Venturi effect of the nebulizing gas at the capillary tip. Therefore, a pressure of 1 psi was selected as optimal. We also examined the inﬂuence of the ﬂow rate of drying gas, which was studied over the range  $1{-}6\;\mathrm{L}/\mathrm{min};$  based on the results, the best sensitivity was achieved with  $3\;\mathrm{L/min}$ . Finally, the inﬂuence of the fragmenter voltage on the ionization process was studied over the range 60–120 V and a value of  $70\,\mathrm{V}$  was selected.

### 3.2 MEPS preconcentration

The free fraction of the proposed analytes in human plasma was analyzed by using the microdialysis probe as coupled to MEPS. The microdialysis probe allows the easy separation of the free analytes from the analytes bounded to proteins that cannot pass through the microdialysis membrane. Preli- minary MEPS tests were performed on water samples spiked with a  $100\,\upmu\mathrm{g/L}$  concentration of each standard; then, an aliquot of  $200\,\upmu\mathrm{L}$  was preconcentrated at  $10\,\upmu\mathrm{L}/\mathrm{s}$ . Before  at $(2\times200\,\upmu\mathrm{L}$   and then eluted with methanol four times $100\,\upmu\mathrm{L}/\mathrm{s})$  . The analysis of the four consecutive extractions $(20\,\upmu\mathrm{L}\times4)$  revealed the presence of traces of the analytes in the second and third eluates, but none in the fourth. Based on these results, we reduced the analyte concentrations down to . However, the consecutive eluates still contained the $5\,\upmu\mathrm{g/L}$  analytes in proportions of  $29.3{-}42.3~\%$  with respect to the ﬁrst. This led us to add formic acid to the eluent in order to facilitate quantitative elution. The formic acid altered the distribution of the analytes in the sorbent and eluent. The acid concentrations studied ranged from 5 to  $200\,\mathrm{mM}$  (see Fig. 2). With  $10\,\mathrm{mM}$  formic acid, the second eluate exhibited  $9.8–20\%$  carryover. Based on the results, using  $20\,\mathrm{mM}$  concentration sufﬁced to ensure quantitative elution of the analytes since carryover was reproducible and always less than  $6\%$ .



Then, the effect of  $\mathrm{pH}$  was examined over the range 3.6–9.5, where no signiﬁcant changes in sensitivity were observed. Therefore,  $\mathrm{pH}~7.4$  was chosen in order to avoid differences with plasma  $\mathrm{pH}$ .

The RSD obtained for six spiked water samples under these conditions was high. This was ascribed to errors arising during the analytical procedure (particularly from human manipulation, the low volume of eluent used and capillary conditioning). In order to improve repeatability, we increased the conditioning time from 3 to  $6\,\mathrm{min}$ , which resulted in dramatically reduced RSD values. Also, increas- ing the eluent volume from 20 to  $50\,\upmu\mathrm{L}$  resulted in further improved RSD. Finally, using a syringe pump substantially reduced the need for human intervention and its associated errors as a result. In addition, incorporation of the syringe pump enabled the use of small eluent volumes (down to ) with good RSD values. $25\,\upmu\mathrm{L}$  The above-described procedure was used to examine the effect of coupling the microdialysis probe via a small piece  $(0.5\,\mathrm{cm})$  of Teﬂon tubing. Figure 3 illustrates the coupling of the automated MEPS system by using a syringe pump withacoupledmicrodialysisprobe.Theprobewas immersed in the sample, held in a vial furnished with a glass insert. As expected, preliminary tests showed that the ﬁber was permeable to the target analytes. However, using the above-described procedure unaltered resulted in strongly diminishedsensitivity.Inmicrodialysis,theabsolute recovery (mol/time unit) of a compound from the sample depends on (i) the ‘‘cut-off’’ of the dialysis membrane, (ii) the length of the membrane, (iii) the ﬂow rate of the and  $10\;S_{\mathrm{y/x}}/$ slope, respectively. The former ranged from 10.4 to  $15.2\,\upmu\mathrm{g/L}$  for free concentrations and 0.6 to  $1.6\,\upmu\mathrm{g/L}$  for total concentrations. The RSD values for peak areas and migration times were under 9 and  $1.4\%,$ , respectively, for both methods. As can be seen from the table, preconcentra- tion took  $35\,\mathrm{min}$  with MEPS-microdialysis and only  $2\,\mathrm{min}$  with the MEPS procedure. LODs were markedly better with MEPS than with microdialysis and MEPS in combination (viz. 0.6 versus  $10.4\,\upmu\mathrm{g/L})$ . This testiﬁes to the better suitability of MEPS than microdialysis–MEPS for the determination of total analyte concentrations.



perfusion ﬂuid and (iv) the diffusion coefﬁcient of the compound through the ﬂuid. Because the former two variables had previously been examined and adjusted, we focused on the latter two. First, we examined the effect of the ﬂow rate of perfusion ﬂuid (Milli-Q water) and hence the contact time between the sample and perfusion ﬂuid, and found increasing the contact time between the sample and ﬁber to boost diffusion of the analytes through the perfusion ﬂuid. The contact time could be improved by holding the ﬂow sequentially or by reducing the ﬂow rate. We performed several tests in order to study the effect of the contact time between sample and ﬁber. The best diffusion rates were obtained with sequential loadings of  $5\,\upmu\mathrm{L}$ at  $0.1\,\upmu\mathrm{L}/\mathrm{s}\!+\!60\,\mathrm{s}$  waiting time 40 times. The total sample volume passed through the sorbent was thus  $100\,\upmu\mathrm{L}.$  However, a long time was required that was successfully shortened by using a continuous sequence without breaks at 0.1 mL/s and then repeating the process another time (100 mL 2).

The next step involved modifying the ionic strength of the sample media. For this purpose, NaCl at concentrations ranging from 10 to 200 g/L was added to the samples. The best recov- eries were obtained with 40 g/L. On the other hand, concen- trations above 200 g/L resulted in ﬁber caking and clogging.

The sample treatment used for the determination of total analyte concentrations in human plasma is described in Section 2. The supernatant collected from the plasma follow- ing treatment with trichloroacetic acid and centrifugation was adjusted to pH 7.4 and preconcentrated with MEPS at 2 mL/s, which was slow enough to ensure optimum analyte retention in the C18 bed. Elution was done with 25 mL of eluent. As stated above, this volume was large enough to ensure repetitive results (RSD ranged from 4.5 to 7.6%).

### 3.4 Analysis of human plasma

a) In these samples, the loaded volume was reduced down to . $250\,\upmu\mathsf{L}$  To demonstrate this interaction with proteins, several tests were done on human serum as sample because of their low content in proteins. As can be seen in Table 2, the recoveries for lidocaine, mepivacaine and bupivacaine spanned the ranges 34.4–37.7, 44.7–46 and  $11.7{-}12.7\%$ , respectively. As expected, the recoveries from serum samples exceeded those from plasma samples by effect of the former containing less protein and a larger free fraction of analytes as a result.



Table 3 shows the ﬁgures of merit of the determination of total analyte concentrations in different plasma samples. As can be seen, samples 1, 6 and 7 tested positive for lido- caine. In addition, analytes recoveries were determined in four different samples spiked with a  $10\,\upmu\mathrm{g/L}$  concentration of each analyte. The estimated recoveries ranged from 78 to  $92\%$  for lidocaine, 58.39 to  $68.7\%$  for mepivacaine and 42.5 to  $56.59\%$  for bupivacaine. As expected, drug recoveries were higher than with the free concentration method by effect of bonds with proteins being broken and the resulting free drugs joining the free fraction. We should note that a volume of sample as low as  $250\,\upmu\mathrm{L}$  should sufﬁce for the intended purpose since samples 6 and 7 exhibited no signiﬁcant difference from sample 1, which was analyzed by using  $500\,\upmu\mathrm{L}$ .



The electropherograms of Fig. 4 expose the differences between the free concentration method and the total concentration method. Figure 4A corresponds to an aliquot of a single plasma sample analyzed with the free concen- tration method, both unspiked (1) and spiked with  $1.6\,\mathrm{mg/L}$  lidocaine, and  $1.4\,\mathrm{mg/L}$  mepivacaine and bupivacaine (2).

Figure 4B shows the electropherograms obtained in the total concentration analysis of other aliquots of the same plasma sample, both unspiked (1) and spiked with  $5\,\upmu\mathrm{g/L}$  lidocaine, and  $4.4\,\upmu\mathrm{g/L}$  mepivacaine and bupivacaine (2).

Figure5showstwoselectedelectropherograms obtained by using the total concentration method. Finally, Fig. 5 (1) corresponds to an unspiked human plasma sample aliquot, which tested positive for lidocaine, and Fig. 5 (2) to an aliquot of the same sample spiked with up to  lidocaine. $10\,\upmu\mathrm{g/L}$ 

## 4Concluding remarks

The high potential of the combination MEPS-CE has been demonstrated. MEPS provides the efﬁcient preconcentration of analytes from a small sample volume. In addition, its automatization has been achieved by using a simple syringe pump. The use of a microdialysis needle on-line coupled to the MEPS-syringe had permitted the analysis of free and total drug in human plasma. This simple strategy is then recommended to perform biological studies.

The authors are grateful to Spain’s Ministry of Education and Science for funding this work as a part of Project 2007- 60426. G.M.-C. also wishes to thank the Ministry for the award of a Research Training Fellowship (Grant BES-2005-10603).

The authors have declared no conﬂict of interest.

## 5References

[1] Drewe, J., Rufer, S., Huwyler, J., Ku¨ sters, E., matogr. B 1997, 691, 105–110

[2] Abdel-Rehim, M., Bielenstein, M., Askemark, Y., Tyre- fors, N., Arvidsson, T., J. Chromatogr. B 2000, 741, 175–188.

[3] Kang, L ., Jun, H. W., McCall, J. W., J. Pharm. Biomed. Anal. 1999, 19, 737–745.

[4] Reif, S., Le Corre, P., Dollo, G., Chevanne, F., Le Verge, R., J. Chromatogr. B 1998, 719, 239–244.

[5] Arvidsson, T., Eklund, E., J. Chromatogr. B 1995, 668, 91–98.

[6] Arvidsson, T., Askemark, Y., Halldin, M. M., Biomed. Chromatogr. 1999, 13, 286–292.

[7] Rifai, N., Hsin, O., Hope, T., Sakamoto, M., Ther. Drug Monit. 2001, 23, 182–186.

[8] Altun, Z., Abdel-Rehim, M., Blomberg, L. G., J. Chro- matogr. B 2004, 813, 129–135.

[9] Abdel-Rehim, M., Andersson, M., Portelius, E., Norsten- Ho¨ o¨ g, C., Blomberg, L.G.,, 313–321.

[10] Koster, E. H. M., Wemes, C., Morsink, J. B., de Jong, G. J., J. Chromatogr. B 2000, 739, 175–182.

[11] Abdel-Rehim,M.,Bielenstein,M.,Arvidsson,T., J. Microcol. Sep. 2000, 12, 308–315.

[12] Wynne, P., Hibbert, R., DiFeo, D., Dawes, P., The Column. 2008, 4, 12– 17.

[13] El-Beqqali, A., Abdel-Rehim, M., J. Sep. Sci. 2007, 30, 2501–2505.

[14] Said, R., Hassan, Z., Hassan, M., Abdel-Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2008, 31, 683–694.

[15] Abdel-Rehim, M., Altun, Z., Blomberg, L., J. Mass Spectrom. 2004, 39, 1488–1493.

[16] Abdel-Rehim, M., Hassan, Z., Skansem, P., Hassan, M., J. Liq. Chromatogr. Rel. Technol. 30, 3029–3041.

[17] Abdel-Rehim, M., Skansen, P., Vita, M., Hassan, Z., Blomberg, L., Hassan, M., Anal. Chim. Acta 2005, 539, 35–39.

[18] Vita, M., Skansen, P., Hassan, M., Abdel-Rehim, M., J. Chromatogr. B 2005, 817, 303–307.

[19] Abdel-Rehim, M ., Andersson, L. I., Altun, Z., Blomberg, L. G., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 1725–1736.

[20] El-Beqqali,A.,Kussak,A.,Blomberg,L.,Abdel- Rehim, M., J. Liq. Chromatogr. Rel. Technol. 2007, 30, 575–586.

[21] Abdel-Rehim, M., Dahlgren, M., Blomberg, L., Claude, S., Tabacchi, R., J. Liq. Chromatogr. Rel. Technol. 2006, 29, 2537–2544.

[22] Abdel-Rehim, M ., J. Chromatogr. B 2004, 801, 317–321.

[23] Al-Nouti, Y., Bartlett, M. G., J. Am. Soc. Mass Spectrom. 2002, 13, 928–935.

[24] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[25] Chee, G. L., Wan, T. S. M., J. Chromatogr. B 1993, 612, 172–177.

[26] Wang, Z., Wan, H., Anderson, M. S., Abdel-Rehim, M., Blomberg, L. G., Electrophoresis 2001, 22, 2495–2502.

[27] Geiser, L., Rudaz, S., Veuthey, J. L., Electrophoresis 2003, 24, 3049–3056.

[28] Curcuruto, O., Zaramella, A., Hamdan, M., Turrina, S., Tagliaro, F., Rapid Commun. Mass Spectrom. 1995, 9, 1487–1491.

[29] Lurie, I. S., Hays, P. A., Parker, K., Electrophoresis 2004, 25, 1580–1591.

[30] Tagharo, F., Smith, F. P., Turrina, S., Equisetto, V., Marigo, M., J. Chromatogr. A 1996, 735, 227–235.

[31] Anderson, M. S., Wan, H., Abdel-Rehim, M., Blomberg, L. G., J. Microcol. Sep. 1999, 11, 620–626.

[32] Bretnall, A. E., Clarke, G. S., J. Chromatogr. A 1996, 745, 145–154.

[33] Bjørnsdottir, I ., Hansen, S. H., J. Biochem. Biophys. Methods 1999, 38, 155–161.

[34] Bjørnsdottir, I., Tjørrnelund, J., Hansen, S. H., Electro- phoresis 1998, 19, 2179–2186.

[35] Cherkaoui, S., Geiser, L., Veuthey, J. L., Chromato- graphia 2000, 52, 403–407.

[36] Zhang, J., Lou, Y., Hoogmartens, J., Van Schepdael, A., Electrophoresis 2006, 27, 4827–4835.

[37] Cherkaoui,S.,Veuthey,J.L.,Analysis1999,27, 765–771.

[38] Anderson, M. S., Lu, B., Abdel-Rehim, M., Blomberg, S., Blomberg, L. G., Rapid Commun. Mass Spectrom. 2004, 18, 2612–2618.

[39] March, J., Simonet, B. M., Grases, F., J. Chromatogr. B 2001, 757, 247–255.
2090 Fractionated Stereotactic Radiotherapy (FSR) for Meningiomas Involving or Close to the Optic Apparatus R. Pfeffer, D. Alezra, J. Menhel, R. Spiegelmann Sheba Medical Center, Tel-Hashomer, Israel Purpose/Objective(s): To assess the clinical and radiological results of fractionated stereotactic radiation (FSR) in the treatment of meningioma involving or close to the optic apparatus, treated at Sheba Medical Center since beginning our FSR program in November 2002. Patients with either optic nerve sheath meningioma or skull base meningioma situated within  $5\;\mathrm{mm}$  of the optic tract were selected for FSR. Patients with skull base meningioma ${>}5~\mathrm{mm}$  from the optic tract received single fraction radiation (radiosurgery) and are not included in this study.

  

 



Materials/Methods: The treatment protocol was for a total radiation dose of 50.4–54 Gy in daily fractions of 1.8 Gy calculated to the  $90\%$  isodose line. The prescription dose covered the entire GTV and at least $95\%$  of the PTV. The PTV included all GTV with 3 mm margins. The maximum optic nerve dose was limited to 56 Gy. Patients were immobilized in a removable frame and underwent MRI with 1 mm slices and CT planning after image fusion. Planning was performed on a BrainLab system. Treatment was deliv- ered using dynamic arcs (typically 4 arcs) with the M3 micro-multileaf collimator.

Seventy ﬁve consecutive patients were treated until November 2006 and have been followed up for at least six months. Seven- teen patients with optic nerve sheath (ons) meningioma and 58 patients with skull base meningioma (predominant sites: 37 cav- ernous sinus, 6 clinoid, 5 orbital apex, 5 petroclival, 5 parasellar) were treated. Age ranged from 26–75 years (median 55 years). All patients with ons meningioma and 19 of the remaining patients presented with visual symptoms. 31 patients with non ons meningiomas had undergone previous surgery. Patients were followed for 6–54 (median 26) months. Follow up was by clinical symptoms, ophthalmologic exam and annual MRI.

Results: Acute toxicity was mild and limited to fatigue (11 patients) headaches (3 patients) and nausea (1 pt) and responded to low dose dexamethasone in all patients. Of the 36 patients with visual symptoms, vision improved in 23  $(64\%)$ , was stable in 11  $(30\%)$  and worsened in 2 patients. Eleven patients reported subjective visual improvement on completion of their radiotherapy course and in most responding patients improved vision was noted within 6 months of completing radiotherapy. No visual impairment was seen in patients presenting without visual symptoms. Fifty eight patients have undergone MRI one year following treatment, none have shown tumor progression.

Conclusions: Within the limitations of the short follow up of our series, FSR is an effective and safe treatment for patients with optic nerve sheath or skull base meningioma involving or close to the optic pathway.

Author Disclosure: R. Pfeffer, None; D. Alezra, None; J. Menhel, None; R. Spiegelmann, None.

## 2091 Neural Stem Cell (NSC)–Preserving External Beam Radiotherapy of Central Nervous System Malignancies

I. J. Barani, L. W. Cuttino, S. H. Benedict, D. A. Todor, E. A. Bump, Y. Wu, T. D. Chung, W. C. Broaddus Virginia Commonwealth University, Richmond, VA Purpose/Objective(s): Recent discoveries have implicated neural stem cells (NSC) as the source of plasticity and repair in the mature mammalian brain. Treatment-induced NSC dysfunction may lead to observed toxicity. This study evaluates the feasibility of neural stem cell (NSC) preserving external beam radiotherapy.



Materials/Methods: A single computed tomography (CT) dataset depicting a right periventricular lesion was used in this study as this location reﬂects the most problematic geometric arrangement with respect to NSC preservation. Conventional and NSC pre- serving radiotherapy (RT) plans were generated for the same lesion using two clinical scenarios: cerebral metastatic disease and primary high-grade glioma. Disease-speciﬁc target volumes were used. Metastatic disease was conventionally treated with whole- brain radiotherapy (WBRT) to  $3750\,\mathrm{cGy}$  (15 fractions) followed by a single stereotactic radiosurgery (SRS) boost of  $1800\,\mathrm{cGy}$  to gross disease only. High-grade glioma was treated with conventional opposed lateral and anterior superior oblique beams to 4600 cGy (23 fractions) followed by a  $1400\ \mathrm{cGy}$  (7 fractions) boost. NSC preservation was achieved in both scenarios with inverse planned intensity modulated radiotherapy (IMRT) (Fig.).

Results: Cumulative dose reductions of  $65\%$  (metastatic disease) and  $25\%$  (high-grade glioma) to the total volume of the intracra- nial NSC compartments were achieved with NSC-preserving IMRT plans. The reduction of entry and exit dose to NSC niches located contralateral to the target contributed most to NSC preservation.

Conclusions: NSC preservation with current external beam radiotherapy techniques is achievable in context of both metastatic brain disease and high-grade glioma, even when the target is located adjacent to a stem cell compartment. Further investigation with clinical trials is warranted to evaluate whether NSC preservation will result in reduced toxicity.

Author Disclosure: I.J. Barani, None; L.W. Cuttino, None; S.H. Benedict, None; D.A. Todor, None; E.A. Bump, None; Y. Wu, None; T.D. Chung, None; W.C. Broaddus, None.

# 2092 Leukoencephalopathy After Whole Brain Radiation Therapy for Metastatic Brain Tumors

J. Ebi, H. Sato, F. Shishido Fukushima Medical University, Fukushima, Japan Purpose/Objective(s): To estimate the incidence of leukoencephalopathy after whole brain radiation therapy (WBRT) of 30 Gy in 10–11 fractions for metastatic brain tumors.

 



Materials/Methods: We performed a retrospective analysis of 99 patients who received WBRT of  $30\;\mathrm{Gy}$  in 10–11 fractions be- tween April 2001 and December 2005. Of these, 61 patients were followed up with CT/MRI for 1 month or more after the com- pletion of WBRT. After WBRT, 33 patients had tumor recurrence (GroupA), and 28 patients had not (GroupB). Tumor recurrence was deﬁned as the increase of the number of brain tumors, the increase in tumor diameter, and/or the increase in edema. And in GroupB, we evaluated the leukoencephalopathy based on CTCAE v3.0.

Results: Of the 61 patients, 34 were male and 27 were female. The average age was 61.0 years (range 29–89). Primary lesions were lung cancer in 42 patients  $(68.9\%)$ , breast cancer in 16  $(26.2\%)$ , and others.

In Group B, the average age was 64.6 years (range 32–89). Primary lesions were lung cancer in 22 patients  $(78.6\%)$ , breast cancer in  $5\,(17.9\%)$ , and other. The average survival time after the completion of WBRT was 244.5 days (range 52–480). Leukoencephal- opathy developed in 4 of the 28 patients  $(14.3\%)$ . Grade 1 was observed in 1 patient and Grade 3 in 3 patients. Of these 4 patients, the average age was 75.5 years (range 72–79), and the average survival time was 377.5 days (range 318–420). Of all 28 patients in GroupB, 8 patients survived for over 9 months after WBRT, and 4 of them  $(50\%)$  had leukoencephalopathy. On the other hand, mild increases in subarachnoid space and/or mild ventriculomegaly without white matter change were observed in 7 patients.

Conclusions: Our result suggests that leukoencephalopathy develops in  $14.3\%$  of patients who do not have recurrence after WBRT, and the risk of leukoencephalopathy grows especially in long time survivors. The indication of WBRT should be carefully considered.

Author Disclosure: J. Ebi, None; H. Sato, None; F. Shishido, None.
2090 Fractionated Stereotactic Radiotherapy (FSR) for Meningiomas Involving or Close to the Optic Apparatus R. Pfeffer, D. Alezra, J. Menhel, R. Spiegelmann Sheba Medical Center, Tel-Hashomer, Israel Purpose/Objective(s): To assess the clinical and radiological results of fractionated stereotactic radiation (FSR) in the treatment of meningioma involving or close to the optic apparatus, treated at Sheba Medical Center since beginning our FSR program in November 2002. Patients with either optic nerve sheath meningioma or skull base meningioma situated within  $5\;\mathrm{mm}$  of the optic tract were selected for FSR. Patients with skull base meningioma ${>}5~\mathrm{mm}$  from the optic tract received single fraction radiation (radiosurgery) and are not included in this study.

  

 



Materials/Methods: The treatment protocol was for a total radiation dose of 50.4–54 Gy in daily fractions of 1.8 Gy calculated to the  $90\%$  isodose line. The prescription dose covered the entire GTV and at least $95\%$  of the PTV. The PTV included all GTV with 3 mm margins. The maximum optic nerve dose was limited to 56 Gy. Patients were immobilized in a removable frame and underwent MRI with 1 mm slices and CT planning after image fusion. Planning was performed on a BrainLab system. Treatment was deliv- ered using dynamic arcs (typically 4 arcs) with the M3 micro-multileaf collimator.

Seventy ﬁve consecutive patients were treated until November 2006 and have been followed up for at least six months. Seven- teen patients with optic nerve sheath (ons) meningioma and 58 patients with skull base meningioma (predominant sites: 37 cav- ernous sinus, 6 clinoid, 5 orbital apex, 5 petroclival, 5 parasellar) were treated. Age ranged from 26–75 years (median 55 years). All patients with ons meningioma and 19 of the remaining patients presented with visual symptoms. 31 patients with non ons meningiomas had undergone previous surgery. Patients were followed for 6–54 (median 26) months. Follow up was by clinical symptoms, ophthalmologic exam and annual MRI.

Results: Acute toxicity was mild and limited to fatigue (11 patients) headaches (3 patients) and nausea (1 pt) and responded to low dose dexamethasone in all patients. Of the 36 patients with visual symptoms, vision improved in 23  $(64\%)$ , was stable in 11  $(30\%)$  and worsened in 2 patients. Eleven patients reported subjective visual improvement on completion of their radiotherapy course and in most responding patients improved vision was noted within 6 months of completing radiotherapy. No visual impairment was seen in patients presenting without visual symptoms. Fifty eight patients have undergone MRI one year following treatment, none have shown tumor progression.

Conclusions: Within the limitations of the short follow up of our series, FSR is an effective and safe treatment for patients with optic nerve sheath or skull base meningioma involving or close to the optic pathway.

Author Disclosure: R. Pfeffer, None; D. Alezra, None; J. Menhel, None; R. Spiegelmann, None.

## 2091 Neural Stem Cell (NSC)–Preserving External Beam Radiotherapy of Central Nervous System Malignancies

I. J. Barani, L. W. Cuttino, S. H. Benedict, D. A. Todor, E. A. Bump, Y. Wu, T. D. Chung, W. C. Broaddus Virginia Commonwealth University, Richmond, VA Purpose/Objective(s): Recent discoveries have implicated neural stem cells (NSC) as the source of plasticity and repair in the mature mammalian brain. Treatment-induced NSC dysfunction may lead to observed toxicity. This study evaluates the feasibility of neural stem cell (NSC) preserving external beam radiotherapy.



Materials/Methods: A single computed tomography (CT) dataset depicting a right periventricular lesion was used in this study as this location reﬂects the most problematic geometric arrangement with respect to NSC preservation. Conventional and NSC pre- serving radiotherapy (RT) plans were generated for the same lesion using two clinical scenarios: cerebral metastatic disease and primary high-grade glioma. Disease-speciﬁc target volumes were used. Metastatic disease was conventionally treated with whole- brain radiotherapy (WBRT) to  $3750\,\mathrm{cGy}$  (15 fractions) followed by a single stereotactic radiosurgery (SRS) boost of  $1800\,\mathrm{cGy}$  to gross disease only. High-grade glioma was treated with conventional opposed lateral and anterior superior oblique beams to 4600 cGy (23 fractions) followed by a  $1400\ \mathrm{cGy}$  (7 fractions) boost. NSC preservation was achieved in both scenarios with inverse planned intensity modulated radiotherapy (IMRT) (Fig.).

Results: Cumulative dose reductions of  $65\%$  (metastatic disease) and  $25\%$  (high-grade glioma) to the total volume of the intracra- nial NSC compartments were achieved with NSC-preserving IMRT plans. The reduction of entry and exit dose to NSC niches located contralateral to the target contributed most to NSC preservation.

Conclusions: NSC preservation with current external beam radiotherapy techniques is achievable in context of both metastatic brain disease and high-grade glioma, even when the target is located adjacent to a stem cell compartment. Further investigation with clinical trials is warranted to evaluate whether NSC preservation will result in reduced toxicity.

Author Disclosure: I.J. Barani, None; L.W. Cuttino, None; S.H. Benedict, None; D.A. Todor, None; E.A. Bump, None; Y. Wu, None; T.D. Chung, None; W.C. Broaddus, None.

# 2092 Leukoencephalopathy After Whole Brain Radiation Therapy for Metastatic Brain Tumors

J. Ebi, H. Sato, F. Shishido Fukushima Medical University, Fukushima, Japan Purpose/Objective(s): To estimate the incidence of leukoencephalopathy after whole brain radiation therapy (WBRT) of 30 Gy in 10–11 fractions for metastatic brain tumors.

 



Materials/Methods: We performed a retrospective analysis of 99 patients who received WBRT of  $30\;\mathrm{Gy}$  in 10–11 fractions be- tween April 2001 and December 2005. Of these, 61 patients were followed up with CT/MRI for 1 month or more after the com- pletion of WBRT. After WBRT, 33 patients had tumor recurrence (GroupA), and 28 patients had not (GroupB). Tumor recurrence was deﬁned as the increase of the number of brain tumors, the increase in tumor diameter, and/or the increase in edema. And in GroupB, we evaluated the leukoencephalopathy based on CTCAE v3.0.

Results: Of the 61 patients, 34 were male and 27 were female. The average age was 61.0 years (range 29–89). Primary lesions were lung cancer in 42 patients  $(68.9\%)$ , breast cancer in 16  $(26.2\%)$ , and others.

In Group B, the average age was 64.6 years (range 32–89). Primary lesions were lung cancer in 22 patients  $(78.6\%)$ , breast cancer in  $5\,(17.9\%)$ , and other. The average survival time after the completion of WBRT was 244.5 days (range 52–480). Leukoencephal- opathy developed in 4 of the 28 patients  $(14.3\%)$ . Grade 1 was observed in 1 patient and Grade 3 in 3 patients. Of these 4 patients, the average age was 75.5 years (range 72–79), and the average survival time was 377.5 days (range 318–420). Of all 28 patients in GroupB, 8 patients survived for over 9 months after WBRT, and 4 of them  $(50\%)$  had leukoencephalopathy. On the other hand, mild increases in subarachnoid space and/or mild ventriculomegaly without white matter change were observed in 7 patients.

Conclusions: Our result suggests that leukoencephalopathy develops in  $14.3\%$  of patients who do not have recurrence after WBRT, and the risk of leukoencephalopathy grows especially in long time survivors. The indication of WBRT should be carefully considered.

Author Disclosure: J. Ebi, None; H. Sato, None; F. Shishido, None.
# © 2016 Association of  Asia Scholars SAGE Publications sagepub.in/home.nav DOI: 10.1177/0976399615624068 http://mla.sagepub.com

E. E. Sajor, B. P. Resurreccion and S. K. Rakshit (Eds). 2014. Bio-innovation  and Poverty Alleviation: Case Studies from Asia. New Delhi: SAGE  Publications Pvt. Ltd, pp. xli + 396, \`1,100. ISBN: 978-81-321-1972-2 (HB).

Poverty has been a long-standing issue in Asia despite many strategies adopted   to tackle it. In this context, the role of emerging bio-technology and bio-  innovations is an interesting area which this book attends to with case studies  from across Asia. It highlights the common claim that new technologies, includ- ing bio-innovations, will ultimately benefit the poor but it argues that making  bio-innovations work for the poor requires that the poor should not only be given  the option to buy and use or not use such technologies but should also be allowed  to participate in the shaping of bio-technology development agenda based on   their own framing of issues and knowledge.

The book is divided into three sections with each one being attended by four to  five papers on the issue. The first part after the introduction by the editor, deals  with assessing the actual direct and potential benefits of bio-innovations for the  poor wherein the individual papers examine cases of water filters in the Philippines,  edible mushrooms in China, aquaculture in Nepal, improved vegetable production  and lazy gardens in Thailand for their poverty reduction impact. The bio-sand  water filter led to economic as well as health benefits for the poor households  which are assessed in monetary terms at the household level. The paper also   examines the willingness to pay and its adoption and identifies factors in social  acceptability of the product. It finally undertakes a social cost–benefits analysis of  the technology and claims that this could be an important mechanism for alleviat- ing poverty. In the case of edible mushrooms in China, the small scale mushroom  production with family labour and linkage to supermarkets helped household  poverty reduction. The mushroom income accounted for 80 per cent of the house- hold income in the study area. The improved technology made use of saw dust  innovated by farmers at the grass-root level. Most of the mushroom was sold at the  local level with some being sold at agricultural produce trading centres. More  importantly, this was supported by the government in terms of subsidies, technical  support, and management and marketing support. On the other hand, commerciali- zation of aquaculture in Nepal showed that there was more reliable and increased  level of production because of adoption at small scale, and farmers obtained addi- tional income from other activities in the pond like vegetables. The innovation was  particularly useful for women providing them self-employment opportunity as  well as additional income. It also led to improved nutrition and empowerment of  women farmers.



In the context of northern Thailand, the two innovations of improved produc- tion of vegetable and lazy gardens are assessed for their pro-poor, gender sensitive  and resilience building impacts, respectively. In the vegetable production, there  was use of bio-inputs, but the adoption was not very high because farmers   still wanted to maintain high level of production and did not have proper market- ing arrangements. But, farmers appreciated the direct access to the market for  organic vegetables and the stability of demand besides lower cost of production  and higher employment. The farmers were getting more household income from  organic vegetable crops. The paper recognizes the improvement brought in by  organic vegetable production in the livelihoods of women farmers, and suggests  the role of agricultural extension in disseminations of bio-innovations. On the  other hand, the lazy garden case study defines them as somewhere between home  garden and agro forestry. It identifies the profile of such gardens, their owners,  workers, and cost and benefits of such enterprises. The lazy gardens use plots of  land in uplands, secondary forest or old Swidden fallows to cultivate a garden of  vegetables and crops without the use of chemical inputs for self-sufficiency.  Complimentary planting is a dominant practice in the system as fruits and vegeta- bles are planted among trees growing naturally. The farmers of lazy gardens   are more educated than their counterparts and households with larger land hold- ings were more likely to have lazy gardens. Women mostly compliment men in  various activities but many activities like planning, land preparation, care of  domestic animals and plants, harvesting and selling are jointly done by men   and women. The main reasons for adopting such gardens were reported to be  family security, environmental consciousness, suitable land and enjoyment.   Major activities having cash income in such gardens included paddy rice and  livestock activities compared with their counterparts who were more into vegeta- ble and fruit activities. However, lack of security of tenure and financial difficul- ties were the major weaknesses of such gardens reported by the farmers. The  paper concludes that such gardens make a major contribution to building social  and ecological resilience of upland farming system and households, and therefore,  they are pro-poor.

The second section of the book highlights the absence of positive impacts of  various bio-innovations across the various Asian contexts and the institutional  constraints therein. The first paper in this section examines the impact and social  differentiation created by commercial shrimp farming in Vietnam wherein a new  method of shrimp farming (pro-biotics) was introduced. The shrimp farming did  help those who were already well off to further consolidate their economic advan- tage and to establish their superior standing within the village economy. The  larger shrimp farms were the most successful in shrimp farming. There were two  dozen new bio-technology products supplied by different companies which were  used in shrimp farming. There was also a gender dimension to it, in that only male  members were in-charge of using bio-innovation products. This led to a new  wealthy class who could afford to send their children to overseas universities and  had high lifestyles with higher involvement in the commercial sex industry and  lifestyle diseases associated with drinking and other leisure activity. The other  interesting paper examines biochar stoves in Cambodia and India. It examines the  stove testers’ perceptions of the gasification stove wherein all the stoves had  higher rating by the testers for safety in comparison to the baseline stoves.   The major problems with the product included inconvenient height for Indian  women who cook while sitting, no easy way to add wood when it is on, require- ment of small pieces of bio-char, unsuitable for large pot cooking, and difficulty  of removing ash from the stove without turning it over. The paper recommends  sociological, anthropological and marketing research to make the product locally  relevant for the poor. There could be many alternative traditional methods to  produce biochar which need to be explored.



The third paper in this section examines the use of vaccine Research &  Development and its patenting for public health in Thailand. It examines the   use of medicine patent tool in bio-medicines. It emphasises the need for political  will for successful adoption of innovative approaches to address Intellectual  Property Rights issues in medicines for human health. The bio-gas programme in  Vietnam is assessed for its impact on the poor in the next chapter with a focus   on institutional factors that constraint the benefits of bio-gas. It estimates the   cost of bio-gas-related infrastructure and the physical, natural, financial, human,  and social capital context of the use of bio-gas and finds that adopters had   high dependence on dairy animals as source of their income compared with   non-adopters who were also dependent on crops and vegetables besides dairy  animals. Most of the households reported savings 100 per cent fuel expenditure  and 20–30 per cent of animal dung for bio-gas. The major environmental benefits  included improved hygiene and clean energy for cooking and reduction in animal  waste. But, the villagers also reported negative impacts of bio-gas on the local  environment and there was not much difference between adopters and non-  adopters. Major constraining factors included bio-gas smell, and dung waste  released in the common sewage from the bio-gas plant. There was also no linkage  between bio-gas programme and the local institutions. This was also constrained  by a flat rate subsidy to all the households.

The last paper in the section examines the poverty alleviation potential of   bio-fertilizers in the Philippines based on a survey of 75 bio-fertilizer adopters  and equal number of non-adopters. It was found that ineffective institutional  arrangement and unclear policies led to poor adoption of the technology besides  poor product quality and lack of marketing. The paper argues for systems  innovation to enhance response of domain actors to the challenges of innovations.  The product availability needs to be improved with the help of social dynamics  and institutional processes for these innovations. It argues for exploring business  tie-up with private companies for marketing and role of intermediaries in the  innovation process.

The third and final section of the book dwells on pro-poor drivers of bioinno- vations and their embedding in poverty alleviation. The paper on transmission of  knowledge in natural farming in north Thailand examines the community sup- ported agriculture (CSA). The CSA farm was funded by a group of community  members who paid an annual fee in exchange for a weekly supply of farm fresh  produce or other products and this helped local farmers increase cash flow and  diversify risk over multiple crops. There were five families in this CSA and 50  consumer families. The CSA network was a significant source of money for the  farmers which came from the sale of food products, and provided a livelihood  security as sometimes advance payment was made to the farmers by the consum- ers. Some of the families also donated their boxes to a boarding school during  short school holidays and farmers continued to get regular income and children  received nutritious organic food. The farmers supplied what was ready and was  accepted by the consumers which led to little wastage and no unmet orders. This  sustainable farming practice also led to more employment and ensured constant  availability of healthy organic food for the farmers’ own families. The farmers  also got a niche in the emerging natural food market. However, this initiative suf- fered from a lack of capacity at the farmer level and competition from mainstream  industrial food system. The paper ends by raising some of the upcoming issues  with the initiative like pricing and similar products in other markets, including  supermarkets, and the need to multiply such initiatives.



The next paper examines the changing trends of bio-innovations in the phar- maceutical industry focusing on issues of inclusion/exclusion of the poor in the  context of South Korea. It examines the case of a drug meant to treat leukaemia.  The paper describes the many stakeholders and their perspective on the product.  The innovators insisted that intellectual property protection was must to promote  innovations and contested that the price set by the Korean government was lower  than the market price. The paper also brings in the Korean government extending  health insurance cover for the new drug and the patent holder stepping in to shoul- der half of the cost of the patient’s payment and refund of payment in cash. As a  result of this, the drug was then available to the Korean patients free of charge.  This was possible due to the efforts of civil organizations which questioned the  public health-related aspects and the international trade conditions to protect  citizen interest.

Another paper in this section focuses on the familiar case of Bt cotton   for its impact on poverty, in areas of China where Bt cotton accounted for 90 per  cent of the sown cotton area. Its adoption led to reduction of production cost   and yield increase as direct impacts and ecological and health benefits as indirect  effects though they were still not certain. Also, large farmers seemed to have   benefitted more from Bt cotton because of their higher education and awareness  level. But, at the same time, state policy towards cotton, competing crops, migra- tion of labour and feminization of agriculture led to decline in the area under  cotton. The paper concludes that certain preconditions like participatory geneti- cally modified organisms (GMOs) policies, context relevant breeding of crops  and varieties, thorough testing, extension and cost-effective access to the techno-  logy are necessary to make the technology pro-poor.

The last paper of the volume examines bio-fertilizer based bio-innovations   in the Indian states of Madhya Pradesh and Tamil Nadu for soyabean and rice  paddy, respectively. The bio-fertilizer promotions were undertaken by a public  institution to make available affordable products to the farmers at local level. But,  the direct focus on poverty was missing in both the innovations, and therefore, it  could not benefit the poor. The specific institutional arrangements and mecha- nisms were not in place to achieve the poverty focus. There was also lack of trust  among the agricultural research community about bio-technology. The paper  argues for focus on specific social groups, institutional support to local commu- nity groups, engagement of actors, partnerships, access to technology, capacity  building of recipients, financial support, and focus on both institutional and tech- nological mechanisms.



The editors have done an excellent job of putting together papers on various  types of bio-innovations from across Asia and classified them in three different  categories in terms of what worked, what did not work and what needs to be done  to make bio-innovations inclusive and helpful in poverty reduction. Some of the  papers offer great insights for rural marketing and learnings on technology adop- tion in rural settings. It would have added much value if editors had also brought  together various lessons from bio-innovation successes and failures at the end of  the volume in a concluding chapter. Despite this, the book is a must read for those  involved in research into and practice of pro-poor interventions in rural areas of  the developing world.

Sukhpal Singh Associate Professor, Centre for Management in Agriculture   Indian Institute of Management, Ahmadabad (India) sukhpal@iimahd.ernet.in

T. Haque, Mandira Bhattacharya and Ankita Goyal. 2015. Socio-economic  Impact Assessment of Bt Cotton in India, New Delhi: Concept Publishing  Company Pvt. Ltd., pp. viii + 146. \`500. ISBN: 978-93-5125-113-2 (HB). DOI: 10.1177/0976399616643798

Cotton has always been one of the most important crops in India. In 2002–2003,  a paradigm shift occurred in Indian cotton farming system after the introduction  of genetically modified (GM) or Bacillus thuringiensis (Bt) cotton for commer- cial cultivation to control American Bollworm damage; that led to two major  changes that paid off for Indian economy as tremendous growth of cotton produc- tion occurred between 2002 and 2012 which led to India’s relative share in global  cotton production increasing from 13 per cent in the early 2000s to 23 per cent by  2012–2013. Though the other issues like increased use of pesticides, Bollworm  resistance to Bt and high seed costs were associated with Bt cotton, yield plate-  aued nationally and farmer suicides increased in some areas recently. Economic  viability of Bt cotton in Indian conditions and linkage of farmers’ poor situation  to adoption of Bt cotton is still not clear. A large number of research studies have  been carried out on the issue of impact of Bt cotton, but confusion remains. Thus,  this book is timely as it provides comparative analysis of socio-economic impact  of Bt cotton in India with before and after methodology. It consists of eight   chapters including introduction and conclusion.
# © 2016 Association of  Asia Scholars SAGE Publications sagepub.in/home.nav DOI: 10.1177/0976399615624068 http://mla.sagepub.com

E. E. Sajor, B. P. Resurreccion and S. K. Rakshit (Eds). 2014. Bio-innovation  and Poverty Alleviation: Case Studies from Asia. New Delhi: SAGE  Publications Pvt. Ltd, pp. xli + 396, \`1,100. ISBN: 978-81-321-1972-2 (HB).

Poverty has been a long-standing issue in Asia despite many strategies adopted   to tackle it. In this context, the role of emerging bio-technology and bio-  innovations is an interesting area which this book attends to with case studies  from across Asia. It highlights the common claim that new technologies, includ- ing bio-innovations, will ultimately benefit the poor but it argues that making  bio-innovations work for the poor requires that the poor should not only be given  the option to buy and use or not use such technologies but should also be allowed  to participate in the shaping of bio-technology development agenda based on   their own framing of issues and knowledge.

The book is divided into three sections with each one being attended by four to  five papers on the issue. The first part after the introduction by the editor, deals  with assessing the actual direct and potential benefits of bio-innovations for the  poor wherein the individual papers examine cases of water filters in the Philippines,  edible mushrooms in China, aquaculture in Nepal, improved vegetable production  and lazy gardens in Thailand for their poverty reduction impact. The bio-sand  water filter led to economic as well as health benefits for the poor households  which are assessed in monetary terms at the household level. The paper also   examines the willingness to pay and its adoption and identifies factors in social  acceptability of the product. It finally undertakes a social cost–benefits analysis of  the technology and claims that this could be an important mechanism for alleviat- ing poverty. In the case of edible mushrooms in China, the small scale mushroom  production with family labour and linkage to supermarkets helped household  poverty reduction. The mushroom income accounted for 80 per cent of the house- hold income in the study area. The improved technology made use of saw dust  innovated by farmers at the grass-root level. Most of the mushroom was sold at the  local level with some being sold at agricultural produce trading centres. More  importantly, this was supported by the government in terms of subsidies, technical  support, and management and marketing support. On the other hand, commerciali- zation of aquaculture in Nepal showed that there was more reliable and increased  level of production because of adoption at small scale, and farmers obtained addi- tional income from other activities in the pond like vegetables. The innovation was  particularly useful for women providing them self-employment opportunity as  well as additional income. It also led to improved nutrition and empowerment of  women farmers.



In the context of northern Thailand, the two innovations of improved produc- tion of vegetable and lazy gardens are assessed for their pro-poor, gender sensitive  and resilience building impacts, respectively. In the vegetable production, there  was use of bio-inputs, but the adoption was not very high because farmers   still wanted to maintain high level of production and did not have proper market- ing arrangements. But, farmers appreciated the direct access to the market for  organic vegetables and the stability of demand besides lower cost of production  and higher employment. The farmers were getting more household income from  organic vegetable crops. The paper recognizes the improvement brought in by  organic vegetable production in the livelihoods of women farmers, and suggests  the role of agricultural extension in disseminations of bio-innovations. On the  other hand, the lazy garden case study defines them as somewhere between home  garden and agro forestry. It identifies the profile of such gardens, their owners,  workers, and cost and benefits of such enterprises. The lazy gardens use plots of  land in uplands, secondary forest or old Swidden fallows to cultivate a garden of  vegetables and crops without the use of chemical inputs for self-sufficiency.  Complimentary planting is a dominant practice in the system as fruits and vegeta- bles are planted among trees growing naturally. The farmers of lazy gardens   are more educated than their counterparts and households with larger land hold- ings were more likely to have lazy gardens. Women mostly compliment men in  various activities but many activities like planning, land preparation, care of  domestic animals and plants, harvesting and selling are jointly done by men   and women. The main reasons for adopting such gardens were reported to be  family security, environmental consciousness, suitable land and enjoyment.   Major activities having cash income in such gardens included paddy rice and  livestock activities compared with their counterparts who were more into vegeta- ble and fruit activities. However, lack of security of tenure and financial difficul- ties were the major weaknesses of such gardens reported by the farmers. The  paper concludes that such gardens make a major contribution to building social  and ecological resilience of upland farming system and households, and therefore,  they are pro-poor.

The second section of the book highlights the absence of positive impacts of  various bio-innovations across the various Asian contexts and the institutional  constraints therein. The first paper in this section examines the impact and social  differentiation created by commercial shrimp farming in Vietnam wherein a new  method of shrimp farming (pro-biotics) was introduced. The shrimp farming did  help those who were already well off to further consolidate their economic advan- tage and to establish their superior standing within the village economy. The  larger shrimp farms were the most successful in shrimp farming. There were two  dozen new bio-technology products supplied by different companies which were  used in shrimp farming. There was also a gender dimension to it, in that only male  members were in-charge of using bio-innovation products. This led to a new  wealthy class who could afford to send their children to overseas universities and  had high lifestyles with higher involvement in the commercial sex industry and  lifestyle diseases associated with drinking and other leisure activity. The other  interesting paper examines biochar stoves in Cambodia and India. It examines the  stove testers’ perceptions of the gasification stove wherein all the stoves had  higher rating by the testers for safety in comparison to the baseline stoves.   The major problems with the product included inconvenient height for Indian  women who cook while sitting, no easy way to add wood when it is on, require- ment of small pieces of bio-char, unsuitable for large pot cooking, and difficulty  of removing ash from the stove without turning it over. The paper recommends  sociological, anthropological and marketing research to make the product locally  relevant for the poor. There could be many alternative traditional methods to  produce biochar which need to be explored.



The third paper in this section examines the use of vaccine Research &  Development and its patenting for public health in Thailand. It examines the   use of medicine patent tool in bio-medicines. It emphasises the need for political  will for successful adoption of innovative approaches to address Intellectual  Property Rights issues in medicines for human health. The bio-gas programme in  Vietnam is assessed for its impact on the poor in the next chapter with a focus   on institutional factors that constraint the benefits of bio-gas. It estimates the   cost of bio-gas-related infrastructure and the physical, natural, financial, human,  and social capital context of the use of bio-gas and finds that adopters had   high dependence on dairy animals as source of their income compared with   non-adopters who were also dependent on crops and vegetables besides dairy  animals. Most of the households reported savings 100 per cent fuel expenditure  and 20–30 per cent of animal dung for bio-gas. The major environmental benefits  included improved hygiene and clean energy for cooking and reduction in animal  waste. But, the villagers also reported negative impacts of bio-gas on the local  environment and there was not much difference between adopters and non-  adopters. Major constraining factors included bio-gas smell, and dung waste  released in the common sewage from the bio-gas plant. There was also no linkage  between bio-gas programme and the local institutions. This was also constrained  by a flat rate subsidy to all the households.

The last paper in the section examines the poverty alleviation potential of   bio-fertilizers in the Philippines based on a survey of 75 bio-fertilizer adopters  and equal number of non-adopters. It was found that ineffective institutional  arrangement and unclear policies led to poor adoption of the technology besides  poor product quality and lack of marketing. The paper argues for systems  innovation to enhance response of domain actors to the challenges of innovations.  The product availability needs to be improved with the help of social dynamics  and institutional processes for these innovations. It argues for exploring business  tie-up with private companies for marketing and role of intermediaries in the  innovation process.

The third and final section of the book dwells on pro-poor drivers of bioinno- vations and their embedding in poverty alleviation. The paper on transmission of  knowledge in natural farming in north Thailand examines the community sup- ported agriculture (CSA). The CSA farm was funded by a group of community  members who paid an annual fee in exchange for a weekly supply of farm fresh  produce or other products and this helped local farmers increase cash flow and  diversify risk over multiple crops. There were five families in this CSA and 50  consumer families. The CSA network was a significant source of money for the  farmers which came from the sale of food products, and provided a livelihood  security as sometimes advance payment was made to the farmers by the consum- ers. Some of the families also donated their boxes to a boarding school during  short school holidays and farmers continued to get regular income and children  received nutritious organic food. The farmers supplied what was ready and was  accepted by the consumers which led to little wastage and no unmet orders. This  sustainable farming practice also led to more employment and ensured constant  availability of healthy organic food for the farmers’ own families. The farmers  also got a niche in the emerging natural food market. However, this initiative suf- fered from a lack of capacity at the farmer level and competition from mainstream  industrial food system. The paper ends by raising some of the upcoming issues  with the initiative like pricing and similar products in other markets, including  supermarkets, and the need to multiply such initiatives.



The next paper examines the changing trends of bio-innovations in the phar- maceutical industry focusing on issues of inclusion/exclusion of the poor in the  context of South Korea. It examines the case of a drug meant to treat leukaemia.  The paper describes the many stakeholders and their perspective on the product.  The innovators insisted that intellectual property protection was must to promote  innovations and contested that the price set by the Korean government was lower  than the market price. The paper also brings in the Korean government extending  health insurance cover for the new drug and the patent holder stepping in to shoul- der half of the cost of the patient’s payment and refund of payment in cash. As a  result of this, the drug was then available to the Korean patients free of charge.  This was possible due to the efforts of civil organizations which questioned the  public health-related aspects and the international trade conditions to protect  citizen interest.

Another paper in this section focuses on the familiar case of Bt cotton   for its impact on poverty, in areas of China where Bt cotton accounted for 90 per  cent of the sown cotton area. Its adoption led to reduction of production cost   and yield increase as direct impacts and ecological and health benefits as indirect  effects though they were still not certain. Also, large farmers seemed to have   benefitted more from Bt cotton because of their higher education and awareness  level. But, at the same time, state policy towards cotton, competing crops, migra- tion of labour and feminization of agriculture led to decline in the area under  cotton. The paper concludes that certain preconditions like participatory geneti- cally modified organisms (GMOs) policies, context relevant breeding of crops  and varieties, thorough testing, extension and cost-effective access to the techno-  logy are necessary to make the technology pro-poor.

The last paper of the volume examines bio-fertilizer based bio-innovations   in the Indian states of Madhya Pradesh and Tamil Nadu for soyabean and rice  paddy, respectively. The bio-fertilizer promotions were undertaken by a public  institution to make available affordable products to the farmers at local level. But,  the direct focus on poverty was missing in both the innovations, and therefore, it  could not benefit the poor. The specific institutional arrangements and mecha- nisms were not in place to achieve the poverty focus. There was also lack of trust  among the agricultural research community about bio-technology. The paper  argues for focus on specific social groups, institutional support to local commu- nity groups, engagement of actors, partnerships, access to technology, capacity  building of recipients, financial support, and focus on both institutional and tech- nological mechanisms.



The editors have done an excellent job of putting together papers on various  types of bio-innovations from across Asia and classified them in three different  categories in terms of what worked, what did not work and what needs to be done  to make bio-innovations inclusive and helpful in poverty reduction. Some of the  papers offer great insights for rural marketing and learnings on technology adop- tion in rural settings. It would have added much value if editors had also brought  together various lessons from bio-innovation successes and failures at the end of  the volume in a concluding chapter. Despite this, the book is a must read for those  involved in research into and practice of pro-poor interventions in rural areas of  the developing world.

Sukhpal Singh Associate Professor, Centre for Management in Agriculture   Indian Institute of Management, Ahmadabad (India) sukhpal@iimahd.ernet.in

T. Haque, Mandira Bhattacharya and Ankita Goyal. 2015. Socio-economic  Impact Assessment of Bt Cotton in India, New Delhi: Concept Publishing  Company Pvt. Ltd., pp. viii + 146. \`500. ISBN: 978-93-5125-113-2 (HB). DOI: 10.1177/0976399616643798

Cotton has always been one of the most important crops in India. In 2002–2003,  a paradigm shift occurred in Indian cotton farming system after the introduction  of genetically modified (GM) or Bacillus thuringiensis (Bt) cotton for commer- cial cultivation to control American Bollworm damage; that led to two major  changes that paid off for Indian economy as tremendous growth of cotton produc- tion occurred between 2002 and 2012 which led to India’s relative share in global  cotton production increasing from 13 per cent in the early 2000s to 23 per cent by  2012–2013. Though the other issues like increased use of pesticides, Bollworm  resistance to Bt and high seed costs were associated with Bt cotton, yield plate-  aued nationally and farmer suicides increased in some areas recently. Economic  viability of Bt cotton in Indian conditions and linkage of farmers’ poor situation  to adoption of Bt cotton is still not clear. A large number of research studies have  been carried out on the issue of impact of Bt cotton, but confusion remains. Thus,  this book is timely as it provides comparative analysis of socio-economic impact  of Bt cotton in India with before and after methodology. It consists of eight   chapters including introduction and conclusion.
The Demographic and Contextual Correlates of Work-Related Repetitive Strain Injuries Among Canadian Men and Women

### F. Curtis Breslin, PhD, S. Ibrahim, MSc, P. Smith, PhD, C. Mustard, ScD,

B. Amick, PhD, and K. Shankardass, PhD

Background The study sought to identify gender differences in work-related repetitive strain injuries (RSI), as well as examine the degree to which non-work factors such as family roles interact with gender to modify RSI risk. Another aim is to examine whether there are potential provincial differences in work-related RSI risk.

Methods The 2003/2005 Canadian Community Health Survey included over 89,000 respondents who reported working in the past 12 months. Separate multi-level models for men and women were used to identify the correlates of work-related RSIs.

Results Women reported sustaining more work-related RSIs than men. Also, having one or more children in the household was associated with lower work-related RSI risk for females. Both men and women in British Columbia reported higher work-related RSI rates than in Ontario.

Conclusions Gender contributes to RSI risk in multiple and diverse ways based on labor market segregation, non-work exposures, and possibly biological vulnerability, which suggests more tailored interventions. Also, the provincial differences indicate that monitoring and surveillance of work injury across jurisdictions can assist in province-wide prevention and occupational health and safety evaluation. Am. J. Ind. Med. 56:1180–1189, 2013. 2013 Wiley Periodicals, Inc.

### KEY WORDS: gender differences; occupational health and safety; occupational injury; geographic; risk factors

## INTRODUCTION

InstituteforWorkandHealth,Toronto,Ontario,Canada SenecaCollege,Schoolof EnglishandLiberalStudies,Toronto,Ontario,Canada Universityof Toronto,DallaLanaSchoolofPublicHealth,Toronto,Ontario,Canada Monash University, School of Public Health and Preventive Medicine, Victoria, Australia 5UniversityofTexasatHoustonSchoolofPublicHealth,SouthwestCenterforOccupa- tional andEnvironmentalHealth,Houston,Texas Departmentof Psychology,WilfridLaurierUniversity,Waterloo,Ontario,Canada 7St.Michael’sHospital,Toronto,Ontario,Canada Contract grant sponsor: Canadian Institutes of Health Research; Contract grantnumber: $\#93981$ . DisclosureStatement:Theauthorsreportnoconflictsofinterests.

Correspondence to:Dr.F.Curtis Breslin,Institute forWork and Health,481University Ave.,Suite800,Toronto,ON,CanadaM5G2E9.E-mail:cbreslin@iwh.on.ca

Women’s participation in work has increased markedly over the last few decades. Currently, Canadian women made up  $47\%$  of total employment numbers [Labour Force Survey,2012].However,gendersegregationinjobs continues to occur in the Canadian labor market [Chung et al., 2000]. For example, when one compares the top ten most common occupations for men and women, only “retail clerk” position appears in both lists [Messing and Stellman, 2006].

Certain types of work-related musculoskeletal (MSK) and repetitive strain injury (RSI) symptoms are reported more frequently by working women than men in industrial- ized countries. For instance, an early review of 21 studies  2013 Wiley Periodicals,Inc. $\copyright$  examining neck and shoulder complaints by occupation and gender showed women having an increased prevalence compared to men [Hagberg and Wegman, 1987]. More recently, women in ofﬁce settings reported more upper extremity complaints than men [Karlqvist et al., 2002; Janwantanakul et al., 2008]. Also lower extremity MSK complaints are higher among women as well [Messing et al., 2008; Messing et al., 2009]. However, less consistent is research examining back complaints, where some studies indicate increased prevalence for women [Krause et al., 1997; de Zwart et al., 1997], while others report increased prevalence for men [Leino-Arjas, 1998; Hooftman et al., 2009].

 



Supporting the notion that work exposures are a major contributing factor for gender differences, some studies examining exposure differences tend to ﬁnd that MSK complaints are not signiﬁcantly different when comparing men and women in the same occupational categories or performing the same tasks [Silverstein et al., 1987; Coury et al., 2002; European Agency for Safety and Health at Work, 2003]. However, a review of speciﬁc exposures and MSK complaints suggests certain vulnerabilities for both men and women [Hooftman et al., 2004]. For example, women completing tasks that required awkward arm postures were at higher risk for neck and shoulder symptoms than men performing similar tasks [Hooftman et al., 2004]. Also, Hooftman and colleagues have found that men were more vulnerable to the effect of lifting than women doing the same activity [Hooftman et al., 2009].

Exposures outside of work such as caring for children and home demands increase the risk of MSK symptoms at work, as well as work absences and work-related disability for women and these non-work exposures are not found to be risk factors for men [McDonough, 1997; Messing et al., 2009; Sandmark, 2009]. This pattern of ﬁndings is consistent with role-strain theory which suggests that the increased and conﬂicting roles of household responsibilities, caregiving, and paid employment have a detrimental effect on women’s health [McDonough, 1997]. It should be noted, however, that there is also support for multiple roles enhancing women’s health [Baruch and Barnett, 1985; Reid and Hardy, 1999]. Role enhancement theory contends that, for women with young children, roles outside the home put them in touch with more social support and economic resources [Rantanen et al., 2011].

The direction and degree of gender differences in working conditions appear to vary by jurisdiction, raising the possibility that broader contextual factors inﬂuence expo- sures and, in turn, work-related RSI risk for men and women. For example, a European study using Karasek’s job demand/ control model found signiﬁcantly lower job demands for men than women in Southern- and Mid-European countries, but the reverse was true in Sweden [de Smet et al., 2005]. In Canada, there are notable legal/regulatory differences between provinces such as required workplace safety committees and such differences may affect the nature/extent of hazard exposure as well as the safety training provided [Tucker, 2003]. Geographic differences in RSI risk may also result from economic policies that affect the industrial mix and socio-economic factors in a jurisdiction [Sullivan, 2000; Breslin et al., 2007]. The aim of this paper is to report on gender differences in work-related RSI, as well as the degree to which non-work factors such as family roles interact with gender to modify RSI risk. Another aim is to examine whether there are potential provincial differences in RSI risk for working men and women.



## METHODS

The Canadian Community Health Survey (CCHS), an ongoing series of annual, cross-sectional surveys, provides estimates of health status and determinants of health at a sub-provinciallevelcalledhealthregions[Statistics Canada, 2003a]. Using a multi-staged, stratiﬁed sampling frame, the CCHS target population consists of household residents aged 12 and older who are living in private dwellings in all Provinces and territories. People living on Indian reserves or Crown lands, residents of institutions, full- time members of the Canadian Armed Forces, and residents of certain remote regions were excluded from the sampling frame. The survey design features and core content have remained largely unchanged during the series of surveys starting in 2001.

The CCHS 2.1 cycle respondents were recruited between January and December of 2003. Out of the 166,222 house- holds selected to participate, an initial response regarding basic demographic information of people dwelling in the household was obtained from 144,836 households  $(87.1\%)$ . Among the 144,836 responding households, one individual per household was randomly selected for an in-depth inter- view. A person-level response rate of  $92.6\%$  was obtained at this stage [Statistics Canada, 2003a].

To increase cell counts at the Census Division level, we also included a subsample of respondents who participated in the CCHS 3.1 (completed two years after CCHS 2.1) into the analytic sample [Statistics Canada, 2006a]. This was appropriate for work injury because the period of 2003 to 2005 was one of relatively stable labor market growth in Canada [Statistics Canada, 2011]. Although the 3.1 survey had the same target population and nearly identical question content, the CCHS 3.1 survey asked only one-third of the respondents the detailed questions about their labor force participation (to reduce respondent burden), so only this portion of respondents who were asked in detail about their employment (men  $\mathrm{n}=8{,}567$  $\mathrm{n}=8{,}450)$  combined with respondents from CCHS 2.1, where all respondents had been asked the detailed questions about labor force participation. The household and person-level response rates for the CCHS 3.1 survey were  $84.9\%$  and  $92.9\%,$ , respectively.



For the purpose of the current analyses, only respondents 15 years or older who had worked at least 1 week in the previous 12 months were included  $(\mathrm{n}=99{,}556)$   $59.4\%$  of the total respondents in that age range. As only a small number of respondents in the Canadian territories (e.g., Northwest Territory) were surveyed  $(\mathrm{n}=3,923)$  after excluding missing values and applying analytic weights to this sample there were  ${<}500$  respondents, these Canadian territory respondents were removed. Of the respondents who met our age and employment inclusion criteria, a further 6,092 respondents were missing information on one or more of the predictor variables and were deleted (i.e.,  $6.4\%$  respondents with missing data), leaving a ﬁnal analytic sample of 89,541 respondents.

Our analysis of missing data showed that the following age groups: 15–24  $\mathrm{{(OR}=0.64;~95\%}$  condence interval  $=(0.59,\ 0.70))$  $\mathrm{OR}=0.52$ ;  $95\%$   $\mathrm{CI}=(0.48,$  –44  $(\mathrm{OR}=0.62;95\%\,\mathrm{CI}=(0.58,0.67))$ – 54 ( $\mathrm{OR}=0.69$ ;  $95\%\ \mathrm{CI}=(0.64,\,0.75))$ cantly less likely to have a missing value than those respondents 55 years and older. Also, those respondents with less than a high school education  $\mathrm{{(OR}=1.74;95\%\,C I}$  those with a high school diploma  $\mathrm{{(OR}}=1.20;95\%\,\mathrm{{CI}}\,(1.11,\$  1.30)) were more likely to have a missing value than those who completed a post secondary degree. In addition, visible minorities  $\mathrm{{OR}}=1.41\$  immigrants who have been in Canada for  ${<}5$  years and those immigrants in Canada for 5 or more years  $\mathrm{{(OR}}=1.7;95\%$   $\operatorname{CI}{(1.40,2.10)};\operatorname{OR}=1.3;95\%\operatorname{CI}{(1.20,1.40)},$  were signiﬁcantly more likely to have a missing value. Also, those respondents working part-time  $\mathrm{{(OR}}=1.5;95\%$   $\mathrm{~CI~}(1.40,\,1.60))$  and those working  $^{<6}$  months out of the year  $\mathrm{(OR}=2.0;~95\%~\mathrm{CI}~(1.90,~2.20))$  to have missing data than full-time workers and those holding jobs the entire year, respectively. The study was approved by the Research Ethics Board of the University of Toronto.

## Measures

## Outcome measure: Work-related repetitive strain injury

To provide a general deﬁnition of RSI for respondents, they were ﬁrst told that RSIs are “caused by overuse or by repeating the same movement frequently (for example, carpal tunnel syndrome, tennis elbow, or tendonitis)” [Statistics Canada, 2005, p. 193]. Next, respondents were asked whether they had any injuries due to repetitive strain which was serious enough to limit their normal activities in the past 12 months. Finally, the respondents were asked, for their most serious RSI in the past 12 months, “what type of activity

[they] were doing when [they] got this repetitive strain?” One of the response options for this question was “working at a job or business (excluding traveling to and from work). We operationalized work-related RSI as only those RSIs of respondents who reported having an RSI in the past 12 months which occurred as a result of working at a job or business.

## Predictors: Socio-demographic variables

A range of socio-demographic variables were included in the analyses. Age was dummy coded into 10-year blocks, with  $^{55+}$  serving as the referent group. Education was categorized based on educational milestones that have been completed, with earning a post-secondary degree being the referent group. Marital status was categorized as married/ cohabitating versus single/divorced/widowed. Family care- taking responsibilities were operationalized as the presence of at least one child in the household who was younger than 12 years old. Immigrant status was categorized as  ${<}5$  years in Canada, 5 years or more in Canada, or not an immigrant (referent group). Another indicator of acculturation included was the respondent’s perception of their ability to converse ﬂuently in English or French, Canada’s two ofﬁcial languages. Visible minority status was obtained in the survey with questions asking about cultural and racial backgrounds. Also included was an indicator of whether the respondent lived in a rural or urban environment, which deﬁned urban using census population counts as a “population concentra- tion of 1,000 or more and population density of 400 or more per square kilometer” [Statistics Canada, 2006b].

## Predictors: Work characteristics

Respondents provided details about the position they considered their main job. Respondents reported on whether they worked full time (i.e.,  $30~\mathrm{hr}$  or more per week) or part- time. They were also asked how many weeks they worked at their main job in the past year. Respondents were also asked whether they were an employee or self-employed. The industrial grouping was derived by asking respondents what kind of business they worked in and providing several examples (e.g., cardboard box manufacturing, retail shoe store). The retail industry was designated as the reference category because it is one of the largest industrial sectors in Canada representing  $11.9\%$  of the working population in 2009 [Statistics Canada, 2010]; and tends to have a lower risk of RSI than other groups [Breslin and Smith, 2006]. With regard to work stress, respondents were asked whether most days in their main job in the past 12 months were stressful on a 5 point Likert scale (i.e., not at all, not very, a bit, quite a bit, or extremely stressful). It was transformed into a three- category variable by collapsing the two lowest and the two highest response categories.

## Predictors: Census division-level variable

Previous studies suggested that area-level variables within a province such as low socioeconomic status of communities may lead to particular vulnerability for injury and disability [Mustard and Frohlich, 1995; McDonough, 1997]. The purpose of including the proportion of single parents in each census division was to operationalize a proxy variable for a potential relevant contextual variable, in this case social/material deprivation. Conceptually, area-level indicators of socioeconomic status can supplement individual indicators of socioeconomic status because “areas with a high level of socioeconomic disadvantage may also be disadvan- taged with respect to transport, retail outlets, leisure facilities, environmental pollution, and social disorganization, in ways that inﬂuence health independently of the socioeconomic characteristic of the people living in these areas” [Smith et al., 1998, p. 399]. Proportion of single parents in an area has been used in previous research and is correlated with other indicators of social/material deprivation [Mustard and Etches, 2003]. Therefore we selected and computed from the 2001 Canadian Census as the percentage of households in a census division who reported being a single parent [Statistics Canada, 2003b]. The census division, the level of aggregation for the present study, is a “general term applying to areas established by provincial law, which are intermediate geographical areas between the municipality (census subdivision) and the province. Usually they are created to facilitate regional planning and the provision of services which can be more effectively delivered on a scale larger than a municipality” [Statistics Canada, 1992].

## Predictors: Provinces

Province referred to the respondent’s province of residence. It was a dummy-coded variable, with Ontario as the referent category. Low cells sizes also led us to aggregate across three Atlantic Provinces (i.e., Newfoundland & Labrador, Nova Scotia, and New Brunswick).

## Analytic Strategy

In the present study, the probability of a work-related RSI was estimated separately for men and women using MLwin 2.0 [Rasbash et al., 2000]. Multiple membership, multi-level logistic regressions were used to estimate the probability of a work-related RSI [Lawson et al., 2003]. This was a Bayesian analysis and the models were ﬁtted using Markov Chain Monte Carlo techniques [Browne, 2009], with simultaneous entry of all predictor variables described above. The unit of observation was the individual respondent (i.e., level 1 of multi-level model), which was nested within census divisions (i.e., level 2, random effect), with provinces included as a ﬁxed, dummy-coded variable.

Spatial dependence, which arises from a geographic area being correlated with neighboring areas, can occur because of the presence of unobserved environmental conditions that are more similar for geographic units that are closer together [Clayton and Hills, 1993]. To control for the inﬂuence of spatial autocorrelation (i.e., spatial random effect), an additional random effect term at the census-division (i.e., subprovincial) level was included in the regression models. Consequently, the regression is described as a multiple membership model because “individuals within an area may be supposed to be inﬂuenced by both an effect of that area and the effects of surrounding contiguous areas” (p. 41) [Fielding and Goldstein, 2006]. In addition, the regressions described below included the cycle that the respondent participated in as a categorical variable in the analyses [Roberts, 2000]. To account for the different probability of selection and nonresponse, we used the survey weights in the descriptive and regression analyses [Lohr, 1999].

To evaluate whether the correlates of work-related RSIs werestatisticallydifferent between the gender- stratiﬁed regressions, we tested for the difference between regression coefﬁcients across men and women for each predictor. We used a method for testing differences between regression coefﬁcients outlined by Allison [1999]. Brieﬂy, this method provides a  $z\cdot$ -test consisting of the difference between the regression coefﬁcients (e.g., children  ${<}12$  years old at homemen’sregression, children  ${<}12$  years old at homewomen’sregression), which form the numerator, and calculates a standard error using the combined standard deviations of the two regression coefﬁcients, which forms the denominator.

## RESULTS

When examining the overall sample with regard to self- reported work-related RSIs, a higher proportion of women reported this type of injury than men  $(7.5\%$  and  $6.9\%,$ , respectively; $\mathrm{OR}=1.1,95\%\,\mathrm{CI}\,(1.04,1.15))$  the frequency of work-related RSI, stratiﬁed by gender for this sample of Canadian workers.

With regard to age, workers 15–24 years old showed the lowest unadjusted rates of work-related RSIs for both men and women. In terms of work variables, full time-workers as well as those reporting working more than 6 months reported the highest unadjusted work-related RSI rates. Also, workers who reported high stress in their jobs exhibited the highest unadjusted work-related RSI rates. For men, the construction industry showed the highest unadjusted rates of work-related RSI rate, while the manufacturing industry showed the highest unadjusted rates of work-related RSI rate for women. Table I also shows that at the provincial level, British Columbia and Saskatchewan show the highest unadjusted rates of work-related RSI for men and women, respectively.

TABLE I. PrevalenceofWork-RelatedRepetitiveStrainInjury(WRKRSI)forDemographic andWorkPredictorsStratifiedbyGender  $(\mathsf{M e n}\!\!:\!\mathsf{n}=45{,}102{;}\mathsf{W o m e n}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! $ 

### MenWomen

### Variable

nWRKRSI, n (%)nWRKRSI, n (%)

## Regression Analyses

The coefﬁcients and adjusted odds ratios for the ﬁnal models are presented in Table II. For work-related RSIs in the past 12 months for men, those in the agriculture/mining/ forestry industry, manufacturing, and the construction industry showed an increased risk of work-related RSIs compared to those men in the retail industry. Men in the arts/ entertainment/accommodation/food service and men in the ﬁnance/real estate/professional services/management and education services industries showed signiﬁcantly decreased risk of a work-related RSI. The association between engaging in full-time work and work-related RSI approached tradition- al levels of signiﬁcance. In terms of provincial differences, men working in British Columbia were signiﬁcantly more likely to report a work-related RSI than in Ontario, even with other demographic and work-related factors controlled. Having dependent children under 12 years old was not associated with work-related RSI for men.

For work-related RSIs in the past 12 months for women, those women having dependent children under 12 years old were at signiﬁcantly lower risk for work-related RSI than those women with no dependent children under 12. Those women in the manufacturing industry showed an increased risk of RSI compared to those in the retail industry. In contrast, those women in the ﬁnance/real estate/professional services/management, health care and social assistance, administration and support/public administration, and edu- cational services showed signiﬁcantly lower risk of work- related RSI. Women engaging in full-time work also showed signiﬁcantly higher risk of work-related RSI than those working part-time. In terms of provincial differences, women working in British Columbia were signiﬁcantly more likely to report a work-related RSI than in Ontario, even with other demographic and work-related factors controlled.



## Statistical examination of gender differences

To examine differences between men and women, we tested for the difference between coefﬁcients across men and women for each predictor, [Allison, 1999]. The  $z$  tests indicated that having a dependent child under 12 years old showed a signiﬁcant difference between gender on this variable’s impact on the risk of work-related RSI (i.e., factor associated with women’s work-related RSIs, but not for men;  $z=2.84$  $P<.011$ ). The agriculture/mining/forestry/utilities ( $z=2.56,$  and construction  $P<.01)$  $(Z=4.38,$  $P<.01)$  industries showed a signiﬁcant difference between genders on the impact of these industries on the risk of work-related

### TABLE II.a(Men:n ¼ ¼

### VariableMen, OR (95% CI)Women, OR (95% CI)

aAdjustedforagegroup,yearofsurvey,maritalstatus,educationlevel,minoritystatus,immigrantstatus,populationdensity,conversinginEnglish/French, $\%$ loneparentin censusdivision,weeksworkedinpast12months,totalhoursworkedinlast12months,workstress,andself-employed.

RSI, with more impact of these on men’s risk. Conversely, men in the arts/entertainment/accommodation/food service industry had a signiﬁcantly lower risk of work-related RSI than women in this industry ( $z=2.10,$  $P<.05)$  resulted in a signiﬁcant between-gender difference. None of the other predictors in the model showed signiﬁcant between- gender differences in their associations with risk of work- related RSI.

## DISCUSSION

As in previous studies, the present study observed at a descriptive level that proportionally more women report work-related RSIs than men [Hagberg and Wegman, 1987]. The present study adds to the literature by being from a representative sample of Canadian workers from all industrial sectors.

The pattern of ﬁndings suggest that gender contributes to RSI risk in multiple and diverse ways based on labor market segregation, non-work exposures, and possibly biological vulnerability. For example, the gendered segregation of the labor market may explain the different industry risk factors for men and women. For men, the construction and agricultural/mining/ﬁshing/forestry industries were associat- ed with elevated work-related RSI risk compared to men in the referent retail industry. However for women, working in theconstructionandagricultural/mining/ﬁshing/forestry industries was not associated with elevated work-related RSI risk compared to women in the retail industry. In addition, compared to their male counterparts, women in the construction and agricultural/mining/ﬁshing/forestry indus- tries showed signiﬁcantly lower work-related RSI risk. Though at this broad level of industrial categorization multiple factors could be involved, this pattern may be related to the different jobs, tasks, and, in turn, hazards that men and women encounter even within the same industry. In contrast, working in the manufacturing industry led to increased work-related RSI risk for both men and women, which may suggest similar hazard exposures that lead to comparable risk.



Another key ﬁnding was that the demographic and non- work-related correlates of RSI differed for men and women. For example, not having one or more dependent children under 12 years old increased work-related RSI for females, but had no signiﬁcant association with men’s work-related RSI risk. Another study examining the impact of having dependent children to care for on work-related back pain showed that women who stood at work, had more than two children, and worked greater than  $40~\mathrm{hr/}$ week were at increased risk for back pain compared to women who stood and did not have this combination of risk factors. Given the differences between the present study and this previous study on the outcome variable and the way the dependent children variable was operationalized and combined with other exposures, this difference in ﬁndings should not be considered contradictory. Instead, the comparison between these two studies may suggest that the inﬂuence of non-work factors such as having dependent children varies depending on the type of work injury and the exposure to other risk factors. In terms of applying the role-enhancement perspec- tive, it is difﬁcult to identify mechanisms whereby caring for dependent children buffers or protects working women from work-related RSIs. A more generic mechanism might be a kind of “healthy worker” effect associated with females engaging in multiple roles. That is, the healthiest women that are least susceptible to work-related RSI, are more likely to take on and maintain both work and child care responsibilities.

Finally, the present study is the ﬁrst to report that men and women in BC show elevated work-related RSI risk compared to their counterparts in Ontario, even with demographic and work-related variables controlled. One potentialexplanationofthisprovincialdifferenceis contextual factors such as differences in OHS legislation, regulations, or enforcement. For instance, cross-provincial analyses of OHS legislation indicated that western provinces such as Alberta had less stringent methods for protecting workers such as rights of participation in workplace safety systems and the right to refuse unsafe work than Ontario [Tucker, 2003]. With regard to British Columbia (BC), however, in the late 1990’s BC instituted ergonomic regulations in the workplace. At ﬁrst glance it appears counter-intuitive that BC put in place stronger measure in protecting workers from RSI, but the work-related RSI risks were higher than other provinces. One way that this pattern may be explained is that the work-related RSI problem in BC was substantially worse, and that the greater work-related RSI burden led to sufﬁcient motivation to pass speciﬁc regulations. Sometimes regulatory initiatives have delayed effects, so a future research question would be to examine more recent work-related RSI trends by province. Another more artifactual explanation is that, through passage of speciﬁc work-related RSI regulations, BC workers as a group were more aware of the issue of work-related RSIs and were, in turn, more willing to attribute RSIs to work causes and/or report work-related RSIs on the survey.



There are several limitations to this study that need to be noted when interpreting the present results. The research design was cross-sectional and observational, so the causal relationships between variables cannot be determined. Another limitation is that the outcome variable was a self- reported measure that was operationalized quite broadly—for example some other studies on gender differences and work- related MSK complaints obtained detail on which site (e.g., neck and shoulder) rather than in the aggregate as in the present study. In addition, neither job title nor a measure of speciﬁc hazard exposures was available in the present health survey. Such potential misspeciﬁcation issues for both the predictors and the outcome variables would tend to reduce the strength of the associations observed. Two strengths of the present study are its representative sample and the use of the same measure across all provinces regarding work- related RSIs.

The implications of these data are that a more tailored, gender speciﬁc approach to prevention is warranted. Also, there appear to be important between-province differences that indicate that monitoring and surveillance of work injury across provinces is an important practical issue for preven- tion, as well as providing an opportunity to compare occupational health and safety initiatives and performance in the aggregate to other provinces. Also, a fruitful area for future research would be the causes of these provincial differences.

## ACKNOWLEDGEMENTS

We would like to acknowledge Albana Çanga for her help in preparing the manuscript and Dr. Karen Messing for her comments on a previous draft. This work was supported by the Canadian Institutes of Health Research grant number #93981. Statistics Canada provided access to the survey data for this study. In terms of competing interests, there are none to declare.

## REFERENCES

Allison P. 1999. Comparing logit and probit coefﬁcients across groups. Sociol Methods Res 28:186–208.

Baruch G, Barnett R. 1985. Role quality, multiple role involvement, and psychological well-being in midlife women. J Pers Soc Psychol 51:578– 585.

Breslin FC, Smith P. 2006. Trial by ﬁre: A multivariate examination of the relation between job tenure and work injuries. Occup Environ Med 63:27–32.

Breslin FC, Smith PM, Dunn JR. 2007. An ecological study of regional variation in work injuries among young workers. BMC Public Health 7:1–12.

Browne WJ. 2009. MCMC estimation in MLwiN Version 2.13. Bristol, UK: Centre for Multilevel Modelling, University of Bristol.

Chung J, Cole D, Clarke J. 2000. Women, work, and injury. In: Sullivan T, editor. Injury and the new world of work. Vancouver: UBC Press, pp. 69–90.

Clayton D, Hills M. 1993. Statistical models in epidemiology. New York: Oxford UP, p. 367.

Coury H, Portcatti I, Alem M, Oishi J. 2002. Inﬂuence of gender on work-related musculoskeletal disorders in repetitive tasks. Int J Ind Ergon 29:33–39.

de Smet P, Sans S, Dramiax M, Boulenguez C, de Backer G, Ferrario M. 2005. Gender and regional differences in perceived job stress across Europe. Eur J Public Health 15:536–545.

de Zwart BC, Broersen JP, Frings-Dresen MH, van Dijk FJ. 1997. Musculoskeletal complaints in The Netherlands in relation to age, gender and physically demanding work. Int Arch Occup Environ Health 70(5):352–360.

European Agency for Safety and Health at Work. 2003. Gender issues in safety and health at work: A review. Luxembourg: Ofﬁce for Ofﬁcial Publications of the. European Communities; http://osha.europa.eu/en/ publications/reports/209 Accessed June 13, 2012.

Fielding A, Goldstein H. Cross-classiﬁed and multiple membership structures in multilevel models: An introduction and review. Birming- ham, UK: University of Birmingham, ISBN 1 84478 797 2.2006.

Hagberg M, Wegman DH. 1987. Prevalence rates and odds ratios of shoulder-neck diseases in different occupational groups. Br J Ind Med 44:602–610.

Hooftman WE, van Poppel MN, van der Beek AJ, Bongers PM, van Mechelen W. 2004. Gender differences in the relations between work- related physical and psychosocial risk factors and musculoskeletal complaints. Scand J Work Environ Health 30(4):261–278.

Hooftman WE, van der Beek AJ, Bongers PM, van Mechelen W. 2009. Is there a gender difference in the effect of work-related physical and psychosocial risk factors on musculoskeletal symptoms and related sickness absence? Scand J Work Environ Health 35(2):85–95.

Janwantanakul P, Pensri P, Jiamjarasrangsri V, Sinsonqsook T. 2008. Prevalence of self-reported musculoskeletal symptoms among ofﬁce workers. Occup Med (Lond) 58(6):436–438.

Karlqvist L, Tornqvist E, Hagberberg M, Hagman M, Toomingas A. 2002. Self-reported working conditions of VDU operators and associations with musculoskeletal symptoms: A cross-sectional study focusing on gender differences. Int J Ind Ergon 30:277–294.

Krause N, Ragland DR, Greiner BA, Fisher JM, Holman DL, Selvin S. 1997. Physical workload and ergonomic factors associated with prevalence of back and neck pain in urban transit operators. Spine 22 (18):2117–2126.

Labour Force Survey. 2012. Labour force characteristics by sex and age group. Ottawa, Ontario: Statistics Canada. http://www.statcan.gc.ca/ tables-tableaux/sum-som/l01/cst01/labor05-eng.htm?sdi¼women Ac- cessed May 14, 2012.

Lawson AB, Browne WJ, Vidal Rodeiro CL. 2003. Disease mapping with WINBUGS and MLwiN. Chichester, UK: Wiley.

Leino-Arjas P. 1998. Smoking and musculoskeletal disorders in the metal industry: A prospective study. Occup Environ Med 55(12):828– 833.

Lohr S. 1999. Sampling: design and analysis. Toronto: Duxbury Press.

McDonough P. 1997. The social patterning of work disability among women in Canada. J Disabil Policy Stud 8:75–95.

Messing K, Stellman J. 2006. Sex, gender, and women’s occupational health: The importance of considering mechanism. Environ Res 101:149–162.

Messing K, Tissot F, Stock S. 2008. Distal lower-extremity pain and work postures in the Quebec population. Am J Public Health 98(4):705– 713.

Messing K, Tissot F, Stock SR. 2009. Should studies of risk factors for musculoskeletal disorders be stratiﬁed by gender? Lessons from the 1998 Quebec Health and Social Survey. Scand J Work Environ Health 35(2):96–112.

Mustard CA, Etches J. 2003. Gender differences in socioeconomic inequality in mortality. J Epidemiol Community Health 57(12):974– 980.

Mustard CA, Frohlich N. 1995. Socioeconomic status and the health of the population. Med Care 33(12 Suppl):DS43–DS54.

Rantanen J, Kinnunen U, Mauno S, Tillemann K. 2011. Introducing theoretical approaches towork-life balance and testing a new typology among professionals. In: Kaiser S, Ringlstetter MJ, Eikhof DR, Pina e Cunha M, editors. Creating Balance? International Perspectives on the Work-Life Integration of Professionals. Berlin: Springer, pp. 27–46.

Rasbash J, Browne W, Goldstein H, Yang M, Plewis I, Healy M, Woodhouse M, Draper G, Langford D, Lewis T. 2000. A user’s guide to MLwiN. Bristol, England: University of Bristol.

Reid J, Hardy M. 1999. Multiple roles and well-being among midlife women: Testing role strain and role enhancement theories. J Gerontol 54B:S329–S338.

Roberts G. 2000. Combining data from the ﬁrst two Panels of SLID for carrying out longitudinal analysis. Unpublished Report. Ottawa, Ontario: Statistics Canada.

Sandmark H. 2009. Job mismatching, unequal opportunities and long- term sickness absence in female white-collar workers in Sweden. Scand J Public Health 37(1):43–49.

Silverstein B, Fine L, Stetson D. 1987. Hand-wrist disorders among investment casting plant workers. J Hand Surg 12A(5 Pt 2):838–844.

Smith GD, Hart C, Watt G, Hole D, Hawthorne V. 1998. Individual social class, area-based deprivation, cardiovascular disease risk factors, and mortality: The Renfrew and Paisley Study. J Epidemiol Community Health 52(6):399–405.

Statistics Canada. 1992. Standard geographical classiﬁcation 1991— The classiﬁcation, Volume 1. Ottawa: Statistics Canada, pp. 153–160.

Statistics Canada. 2003a. Canadian Community Health Survey: User Guide for the Public Use Microdata File. Ottawa,Ontario: Ministry of Industry,82M0013GPE.http://www23.statcan.gc.ca:81/imdb-bmdi/ pub/document/3226_D7_T9_V2-eng.pdf Accessed June 13, 2012.

Statistics Canada. 2003b. Canadian Census for 2001 Dictionary. Catalogue92-378-XIE.http://publications.gc.ca/Collection/Statcan/ 92-378-X/92-378-XIE02002.pdf Accessed June 13, 2012.

Statistics Canada. 2005. Canadian Community Health Survey, Cycle 3.1, Final questionnaire. Ottawa, Ontario: Statistics Canada.

Statistics Canada. 2006a. Canadian Community Health Survey Cycle 3.1 (2005). Public use microdata ﬁle user guide. Ottawa, Ontario: Statistics Canada. http://www23.statcan.gc.ca:81/imdb-bmdi/pub/doc- ument/3226_D7_T9_V3-eng.pdf Accessed June 13, 2012.



Statistics Canada. 2006b. Canadian Community Health Survey, Cycle 3.1, derived variable speciﬁcation. Ottawa, Ontario: Statistics Canada.

Statistics Canada. 2010. Retail sector activity. In: Statistics Canada. State of Retail: The Canadian Report 2010. Ottawa, Ontario: Statistics Canada.http://www.ic.gc.ca/eic/site/retra-comde.,nsf/eng/qn00285. html Statistics Canada. 2011. Labour: Unemployment rate by selected age groups. Ottawa, Ontario: Statistics Canada. http://www.statcan.gc.ca/ pub/11-402-x/2010000/chap/lab-tra/lab-tra-eng.htm,2011Accessed May 14, 2012.



Sullivan T. 2000. Injury and the new world of work. Vancouver: UBC Press, p. 368.

Tucker E. 2003. Diverging trends in worker health and safety protection and participation in Canada, 1985–2000. Ind Relat 58(3):395–423.
The Demographic and Contextual Correlates of Work-Related Repetitive Strain Injuries Among Canadian Men and Women

### F. Curtis Breslin, PhD, S. Ibrahim, MSc, P. Smith, PhD, C. Mustard, ScD,

B. Amick, PhD, and K. Shankardass, PhD

Background The study sought to identify gender differences in work-related repetitive strain injuries (RSI), as well as examine the degree to which non-work factors such as family roles interact with gender to modify RSI risk. Another aim is to examine whether there are potential provincial differences in work-related RSI risk.

Methods The 2003/2005 Canadian Community Health Survey included over 89,000 respondents who reported working in the past 12 months. Separate multi-level models for men and women were used to identify the correlates of work-related RSIs.

Results Women reported sustaining more work-related RSIs than men. Also, having one or more children in the household was associated with lower work-related RSI risk for females. Both men and women in British Columbia reported higher work-related RSI rates than in Ontario.

Conclusions Gender contributes to RSI risk in multiple and diverse ways based on labor market segregation, non-work exposures, and possibly biological vulnerability, which suggests more tailored interventions. Also, the provincial differences indicate that monitoring and surveillance of work injury across jurisdictions can assist in province-wide prevention and occupational health and safety evaluation. Am. J. Ind. Med. 56:1180–1189, 2013. 2013 Wiley Periodicals, Inc.

### KEY WORDS: gender differences; occupational health and safety; occupational injury; geographic; risk factors

## INTRODUCTION

InstituteforWorkandHealth,Toronto,Ontario,Canada SenecaCollege,Schoolof EnglishandLiberalStudies,Toronto,Ontario,Canada Universityof Toronto,DallaLanaSchoolofPublicHealth,Toronto,Ontario,Canada Monash University, School of Public Health and Preventive Medicine, Victoria, Australia 5UniversityofTexasatHoustonSchoolofPublicHealth,SouthwestCenterforOccupa- tional andEnvironmentalHealth,Houston,Texas Departmentof Psychology,WilfridLaurierUniversity,Waterloo,Ontario,Canada 7St.Michael’sHospital,Toronto,Ontario,Canada Contract grant sponsor: Canadian Institutes of Health Research; Contract grantnumber: $\#93981$ . DisclosureStatement:Theauthorsreportnoconflictsofinterests.

Correspondence to:Dr.F.Curtis Breslin,Institute forWork and Health,481University Ave.,Suite800,Toronto,ON,CanadaM5G2E9.E-mail:cbreslin@iwh.on.ca

Women’s participation in work has increased markedly over the last few decades. Currently, Canadian women made up  $47\%$  of total employment numbers [Labour Force Survey,2012].However,gendersegregationinjobs continues to occur in the Canadian labor market [Chung et al., 2000]. For example, when one compares the top ten most common occupations for men and women, only “retail clerk” position appears in both lists [Messing and Stellman, 2006].

Certain types of work-related musculoskeletal (MSK) and repetitive strain injury (RSI) symptoms are reported more frequently by working women than men in industrial- ized countries. For instance, an early review of 21 studies  2013 Wiley Periodicals,Inc. $\copyright$  examining neck and shoulder complaints by occupation and gender showed women having an increased prevalence compared to men [Hagberg and Wegman, 1987]. More recently, women in ofﬁce settings reported more upper extremity complaints than men [Karlqvist et al., 2002; Janwantanakul et al., 2008]. Also lower extremity MSK complaints are higher among women as well [Messing et al., 2008; Messing et al., 2009]. However, less consistent is research examining back complaints, where some studies indicate increased prevalence for women [Krause et al., 1997; de Zwart et al., 1997], while others report increased prevalence for men [Leino-Arjas, 1998; Hooftman et al., 2009].

 



Supporting the notion that work exposures are a major contributing factor for gender differences, some studies examining exposure differences tend to ﬁnd that MSK complaints are not signiﬁcantly different when comparing men and women in the same occupational categories or performing the same tasks [Silverstein et al., 1987; Coury et al., 2002; European Agency for Safety and Health at Work, 2003]. However, a review of speciﬁc exposures and MSK complaints suggests certain vulnerabilities for both men and women [Hooftman et al., 2004]. For example, women completing tasks that required awkward arm postures were at higher risk for neck and shoulder symptoms than men performing similar tasks [Hooftman et al., 2004]. Also, Hooftman and colleagues have found that men were more vulnerable to the effect of lifting than women doing the same activity [Hooftman et al., 2009].

Exposures outside of work such as caring for children and home demands increase the risk of MSK symptoms at work, as well as work absences and work-related disability for women and these non-work exposures are not found to be risk factors for men [McDonough, 1997; Messing et al., 2009; Sandmark, 2009]. This pattern of ﬁndings is consistent with role-strain theory which suggests that the increased and conﬂicting roles of household responsibilities, caregiving, and paid employment have a detrimental effect on women’s health [McDonough, 1997]. It should be noted, however, that there is also support for multiple roles enhancing women’s health [Baruch and Barnett, 1985; Reid and Hardy, 1999]. Role enhancement theory contends that, for women with young children, roles outside the home put them in touch with more social support and economic resources [Rantanen et al., 2011].

The direction and degree of gender differences in working conditions appear to vary by jurisdiction, raising the possibility that broader contextual factors inﬂuence expo- sures and, in turn, work-related RSI risk for men and women. For example, a European study using Karasek’s job demand/ control model found signiﬁcantly lower job demands for men than women in Southern- and Mid-European countries, but the reverse was true in Sweden [de Smet et al., 2005]. In Canada, there are notable legal/regulatory differences between provinces such as required workplace safety committees and such differences may affect the nature/extent of hazard exposure as well as the safety training provided [Tucker, 2003]. Geographic differences in RSI risk may also result from economic policies that affect the industrial mix and socio-economic factors in a jurisdiction [Sullivan, 2000; Breslin et al., 2007]. The aim of this paper is to report on gender differences in work-related RSI, as well as the degree to which non-work factors such as family roles interact with gender to modify RSI risk. Another aim is to examine whether there are potential provincial differences in RSI risk for working men and women.



## METHODS

The Canadian Community Health Survey (CCHS), an ongoing series of annual, cross-sectional surveys, provides estimates of health status and determinants of health at a sub-provinciallevelcalledhealthregions[Statistics Canada, 2003a]. Using a multi-staged, stratiﬁed sampling frame, the CCHS target population consists of household residents aged 12 and older who are living in private dwellings in all Provinces and territories. People living on Indian reserves or Crown lands, residents of institutions, full- time members of the Canadian Armed Forces, and residents of certain remote regions were excluded from the sampling frame. The survey design features and core content have remained largely unchanged during the series of surveys starting in 2001.

The CCHS 2.1 cycle respondents were recruited between January and December of 2003. Out of the 166,222 house- holds selected to participate, an initial response regarding basic demographic information of people dwelling in the household was obtained from 144,836 households  $(87.1\%)$ . Among the 144,836 responding households, one individual per household was randomly selected for an in-depth inter- view. A person-level response rate of  $92.6\%$  was obtained at this stage [Statistics Canada, 2003a].

To increase cell counts at the Census Division level, we also included a subsample of respondents who participated in the CCHS 3.1 (completed two years after CCHS 2.1) into the analytic sample [Statistics Canada, 2006a]. This was appropriate for work injury because the period of 2003 to 2005 was one of relatively stable labor market growth in Canada [Statistics Canada, 2011]. Although the 3.1 survey had the same target population and nearly identical question content, the CCHS 3.1 survey asked only one-third of the respondents the detailed questions about their labor force participation (to reduce respondent burden), so only this portion of respondents who were asked in detail about their employment (men  $\mathrm{n}=8{,}567$  $\mathrm{n}=8{,}450)$  combined with respondents from CCHS 2.1, where all respondents had been asked the detailed questions about labor force participation. The household and person-level response rates for the CCHS 3.1 survey were  $84.9\%$  and  $92.9\%,$ , respectively.



For the purpose of the current analyses, only respondents 15 years or older who had worked at least 1 week in the previous 12 months were included  $(\mathrm{n}=99{,}556)$   $59.4\%$  of the total respondents in that age range. As only a small number of respondents in the Canadian territories (e.g., Northwest Territory) were surveyed  $(\mathrm{n}=3,923)$  after excluding missing values and applying analytic weights to this sample there were  ${<}500$  respondents, these Canadian territory respondents were removed. Of the respondents who met our age and employment inclusion criteria, a further 6,092 respondents were missing information on one or more of the predictor variables and were deleted (i.e.,  $6.4\%$  respondents with missing data), leaving a ﬁnal analytic sample of 89,541 respondents.

Our analysis of missing data showed that the following age groups: 15–24  $\mathrm{{(OR}=0.64;~95\%}$  condence interval  $=(0.59,\ 0.70))$  $\mathrm{OR}=0.52$ ;  $95\%$   $\mathrm{CI}=(0.48,$  –44  $(\mathrm{OR}=0.62;95\%\,\mathrm{CI}=(0.58,0.67))$ – 54 ( $\mathrm{OR}=0.69$ ;  $95\%\ \mathrm{CI}=(0.64,\,0.75))$ cantly less likely to have a missing value than those respondents 55 years and older. Also, those respondents with less than a high school education  $\mathrm{{(OR}=1.74;95\%\,C I}$  those with a high school diploma  $\mathrm{{(OR}}=1.20;95\%\,\mathrm{{CI}}\,(1.11,\$  1.30)) were more likely to have a missing value than those who completed a post secondary degree. In addition, visible minorities  $\mathrm{{OR}}=1.41\$  immigrants who have been in Canada for  ${<}5$  years and those immigrants in Canada for 5 or more years  $\mathrm{{(OR}}=1.7;95\%$   $\operatorname{CI}{(1.40,2.10)};\operatorname{OR}=1.3;95\%\operatorname{CI}{(1.20,1.40)},$  were signiﬁcantly more likely to have a missing value. Also, those respondents working part-time  $\mathrm{{(OR}}=1.5;95\%$   $\mathrm{~CI~}(1.40,\,1.60))$  and those working  $^{<6}$  months out of the year  $\mathrm{(OR}=2.0;~95\%~\mathrm{CI}~(1.90,~2.20))$  to have missing data than full-time workers and those holding jobs the entire year, respectively. The study was approved by the Research Ethics Board of the University of Toronto.

## Measures

## Outcome measure: Work-related repetitive strain injury

To provide a general deﬁnition of RSI for respondents, they were ﬁrst told that RSIs are “caused by overuse or by repeating the same movement frequently (for example, carpal tunnel syndrome, tennis elbow, or tendonitis)” [Statistics Canada, 2005, p. 193]. Next, respondents were asked whether they had any injuries due to repetitive strain which was serious enough to limit their normal activities in the past 12 months. Finally, the respondents were asked, for their most serious RSI in the past 12 months, “what type of activity

[they] were doing when [they] got this repetitive strain?” One of the response options for this question was “working at a job or business (excluding traveling to and from work). We operationalized work-related RSI as only those RSIs of respondents who reported having an RSI in the past 12 months which occurred as a result of working at a job or business.

## Predictors: Socio-demographic variables

A range of socio-demographic variables were included in the analyses. Age was dummy coded into 10-year blocks, with  $^{55+}$  serving as the referent group. Education was categorized based on educational milestones that have been completed, with earning a post-secondary degree being the referent group. Marital status was categorized as married/ cohabitating versus single/divorced/widowed. Family care- taking responsibilities were operationalized as the presence of at least one child in the household who was younger than 12 years old. Immigrant status was categorized as  ${<}5$  years in Canada, 5 years or more in Canada, or not an immigrant (referent group). Another indicator of acculturation included was the respondent’s perception of their ability to converse ﬂuently in English or French, Canada’s two ofﬁcial languages. Visible minority status was obtained in the survey with questions asking about cultural and racial backgrounds. Also included was an indicator of whether the respondent lived in a rural or urban environment, which deﬁned urban using census population counts as a “population concentra- tion of 1,000 or more and population density of 400 or more per square kilometer” [Statistics Canada, 2006b].

## Predictors: Work characteristics

Respondents provided details about the position they considered their main job. Respondents reported on whether they worked full time (i.e.,  $30~\mathrm{hr}$  or more per week) or part- time. They were also asked how many weeks they worked at their main job in the past year. Respondents were also asked whether they were an employee or self-employed. The industrial grouping was derived by asking respondents what kind of business they worked in and providing several examples (e.g., cardboard box manufacturing, retail shoe store). The retail industry was designated as the reference category because it is one of the largest industrial sectors in Canada representing  $11.9\%$  of the working population in 2009 [Statistics Canada, 2010]; and tends to have a lower risk of RSI than other groups [Breslin and Smith, 2006]. With regard to work stress, respondents were asked whether most days in their main job in the past 12 months were stressful on a 5 point Likert scale (i.e., not at all, not very, a bit, quite a bit, or extremely stressful). It was transformed into a three- category variable by collapsing the two lowest and the two highest response categories.

## Predictors: Census division-level variable

Previous studies suggested that area-level variables within a province such as low socioeconomic status of communities may lead to particular vulnerability for injury and disability [Mustard and Frohlich, 1995; McDonough, 1997]. The purpose of including the proportion of single parents in each census division was to operationalize a proxy variable for a potential relevant contextual variable, in this case social/material deprivation. Conceptually, area-level indicators of socioeconomic status can supplement individual indicators of socioeconomic status because “areas with a high level of socioeconomic disadvantage may also be disadvan- taged with respect to transport, retail outlets, leisure facilities, environmental pollution, and social disorganization, in ways that inﬂuence health independently of the socioeconomic characteristic of the people living in these areas” [Smith et al., 1998, p. 399]. Proportion of single parents in an area has been used in previous research and is correlated with other indicators of social/material deprivation [Mustard and Etches, 2003]. Therefore we selected and computed from the 2001 Canadian Census as the percentage of households in a census division who reported being a single parent [Statistics Canada, 2003b]. The census division, the level of aggregation for the present study, is a “general term applying to areas established by provincial law, which are intermediate geographical areas between the municipality (census subdivision) and the province. Usually they are created to facilitate regional planning and the provision of services which can be more effectively delivered on a scale larger than a municipality” [Statistics Canada, 1992].

## Predictors: Provinces

Province referred to the respondent’s province of residence. It was a dummy-coded variable, with Ontario as the referent category. Low cells sizes also led us to aggregate across three Atlantic Provinces (i.e., Newfoundland & Labrador, Nova Scotia, and New Brunswick).

## Analytic Strategy

In the present study, the probability of a work-related RSI was estimated separately for men and women using MLwin 2.0 [Rasbash et al., 2000]. Multiple membership, multi-level logistic regressions were used to estimate the probability of a work-related RSI [Lawson et al., 2003]. This was a Bayesian analysis and the models were ﬁtted using Markov Chain Monte Carlo techniques [Browne, 2009], with simultaneous entry of all predictor variables described above. The unit of observation was the individual respondent (i.e., level 1 of multi-level model), which was nested within census divisions (i.e., level 2, random effect), with provinces included as a ﬁxed, dummy-coded variable.

Spatial dependence, which arises from a geographic area being correlated with neighboring areas, can occur because of the presence of unobserved environmental conditions that are more similar for geographic units that are closer together [Clayton and Hills, 1993]. To control for the inﬂuence of spatial autocorrelation (i.e., spatial random effect), an additional random effect term at the census-division (i.e., subprovincial) level was included in the regression models. Consequently, the regression is described as a multiple membership model because “individuals within an area may be supposed to be inﬂuenced by both an effect of that area and the effects of surrounding contiguous areas” (p. 41) [Fielding and Goldstein, 2006]. In addition, the regressions described below included the cycle that the respondent participated in as a categorical variable in the analyses [Roberts, 2000]. To account for the different probability of selection and nonresponse, we used the survey weights in the descriptive and regression analyses [Lohr, 1999].

To evaluate whether the correlates of work-related RSIs werestatisticallydifferent between the gender- stratiﬁed regressions, we tested for the difference between regression coefﬁcients across men and women for each predictor. We used a method for testing differences between regression coefﬁcients outlined by Allison [1999]. Brieﬂy, this method provides a  $z\cdot$ -test consisting of the difference between the regression coefﬁcients (e.g., children  ${<}12$  years old at homemen’sregression, children  ${<}12$  years old at homewomen’sregression), which form the numerator, and calculates a standard error using the combined standard deviations of the two regression coefﬁcients, which forms the denominator.

## RESULTS

When examining the overall sample with regard to self- reported work-related RSIs, a higher proportion of women reported this type of injury than men  $(7.5\%$  and  $6.9\%,$ , respectively; $\mathrm{OR}=1.1,95\%\,\mathrm{CI}\,(1.04,1.15))$  the frequency of work-related RSI, stratiﬁed by gender for this sample of Canadian workers.

With regard to age, workers 15–24 years old showed the lowest unadjusted rates of work-related RSIs for both men and women. In terms of work variables, full time-workers as well as those reporting working more than 6 months reported the highest unadjusted work-related RSI rates. Also, workers who reported high stress in their jobs exhibited the highest unadjusted work-related RSI rates. For men, the construction industry showed the highest unadjusted rates of work-related RSI rate, while the manufacturing industry showed the highest unadjusted rates of work-related RSI rate for women. Table I also shows that at the provincial level, British Columbia and Saskatchewan show the highest unadjusted rates of work-related RSI for men and women, respectively.

TABLE I. PrevalenceofWork-RelatedRepetitiveStrainInjury(WRKRSI)forDemographic andWorkPredictorsStratifiedbyGender  $(\mathsf{M e n}\!\!:\!\mathsf{n}=45{,}102{;}\mathsf{W o m e n}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! $ 

### MenWomen

### Variable

nWRKRSI, n (%)nWRKRSI, n (%)

## Regression Analyses

The coefﬁcients and adjusted odds ratios for the ﬁnal models are presented in Table II. For work-related RSIs in the past 12 months for men, those in the agriculture/mining/ forestry industry, manufacturing, and the construction industry showed an increased risk of work-related RSIs compared to those men in the retail industry. Men in the arts/ entertainment/accommodation/food service and men in the ﬁnance/real estate/professional services/management and education services industries showed signiﬁcantly decreased risk of a work-related RSI. The association between engaging in full-time work and work-related RSI approached tradition- al levels of signiﬁcance. In terms of provincial differences, men working in British Columbia were signiﬁcantly more likely to report a work-related RSI than in Ontario, even with other demographic and work-related factors controlled. Having dependent children under 12 years old was not associated with work-related RSI for men.

For work-related RSIs in the past 12 months for women, those women having dependent children under 12 years old were at signiﬁcantly lower risk for work-related RSI than those women with no dependent children under 12. Those women in the manufacturing industry showed an increased risk of RSI compared to those in the retail industry. In contrast, those women in the ﬁnance/real estate/professional services/management, health care and social assistance, administration and support/public administration, and edu- cational services showed signiﬁcantly lower risk of work- related RSI. Women engaging in full-time work also showed signiﬁcantly higher risk of work-related RSI than those working part-time. In terms of provincial differences, women working in British Columbia were signiﬁcantly more likely to report a work-related RSI than in Ontario, even with other demographic and work-related factors controlled.



## Statistical examination of gender differences

To examine differences between men and women, we tested for the difference between coefﬁcients across men and women for each predictor, [Allison, 1999]. The  $z$  tests indicated that having a dependent child under 12 years old showed a signiﬁcant difference between gender on this variable’s impact on the risk of work-related RSI (i.e., factor associated with women’s work-related RSIs, but not for men;  $z=2.84$  $P<.011$ ). The agriculture/mining/forestry/utilities ( $z=2.56,$  and construction  $P<.01)$  $(Z=4.38,$  $P<.01)$  industries showed a signiﬁcant difference between genders on the impact of these industries on the risk of work-related

### TABLE II.a(Men:n ¼ ¼

### VariableMen, OR (95% CI)Women, OR (95% CI)

aAdjustedforagegroup,yearofsurvey,maritalstatus,educationlevel,minoritystatus,immigrantstatus,populationdensity,conversinginEnglish/French, $\%$ loneparentin censusdivision,weeksworkedinpast12months,totalhoursworkedinlast12months,workstress,andself-employed.

RSI, with more impact of these on men’s risk. Conversely, men in the arts/entertainment/accommodation/food service industry had a signiﬁcantly lower risk of work-related RSI than women in this industry ( $z=2.10,$  $P<.05)$  resulted in a signiﬁcant between-gender difference. None of the other predictors in the model showed signiﬁcant between- gender differences in their associations with risk of work- related RSI.

## DISCUSSION

As in previous studies, the present study observed at a descriptive level that proportionally more women report work-related RSIs than men [Hagberg and Wegman, 1987]. The present study adds to the literature by being from a representative sample of Canadian workers from all industrial sectors.

The pattern of ﬁndings suggest that gender contributes to RSI risk in multiple and diverse ways based on labor market segregation, non-work exposures, and possibly biological vulnerability. For example, the gendered segregation of the labor market may explain the different industry risk factors for men and women. For men, the construction and agricultural/mining/ﬁshing/forestry industries were associat- ed with elevated work-related RSI risk compared to men in the referent retail industry. However for women, working in theconstructionandagricultural/mining/ﬁshing/forestry industries was not associated with elevated work-related RSI risk compared to women in the retail industry. In addition, compared to their male counterparts, women in the construction and agricultural/mining/ﬁshing/forestry indus- tries showed signiﬁcantly lower work-related RSI risk. Though at this broad level of industrial categorization multiple factors could be involved, this pattern may be related to the different jobs, tasks, and, in turn, hazards that men and women encounter even within the same industry. In contrast, working in the manufacturing industry led to increased work-related RSI risk for both men and women, which may suggest similar hazard exposures that lead to comparable risk.



Another key ﬁnding was that the demographic and non- work-related correlates of RSI differed for men and women. For example, not having one or more dependent children under 12 years old increased work-related RSI for females, but had no signiﬁcant association with men’s work-related RSI risk. Another study examining the impact of having dependent children to care for on work-related back pain showed that women who stood at work, had more than two children, and worked greater than  $40~\mathrm{hr/}$ week were at increased risk for back pain compared to women who stood and did not have this combination of risk factors. Given the differences between the present study and this previous study on the outcome variable and the way the dependent children variable was operationalized and combined with other exposures, this difference in ﬁndings should not be considered contradictory. Instead, the comparison between these two studies may suggest that the inﬂuence of non-work factors such as having dependent children varies depending on the type of work injury and the exposure to other risk factors. In terms of applying the role-enhancement perspec- tive, it is difﬁcult to identify mechanisms whereby caring for dependent children buffers or protects working women from work-related RSIs. A more generic mechanism might be a kind of “healthy worker” effect associated with females engaging in multiple roles. That is, the healthiest women that are least susceptible to work-related RSI, are more likely to take on and maintain both work and child care responsibilities.

Finally, the present study is the ﬁrst to report that men and women in BC show elevated work-related RSI risk compared to their counterparts in Ontario, even with demographic and work-related variables controlled. One potentialexplanationofthisprovincialdifferenceis contextual factors such as differences in OHS legislation, regulations, or enforcement. For instance, cross-provincial analyses of OHS legislation indicated that western provinces such as Alberta had less stringent methods for protecting workers such as rights of participation in workplace safety systems and the right to refuse unsafe work than Ontario [Tucker, 2003]. With regard to British Columbia (BC), however, in the late 1990’s BC instituted ergonomic regulations in the workplace. At ﬁrst glance it appears counter-intuitive that BC put in place stronger measure in protecting workers from RSI, but the work-related RSI risks were higher than other provinces. One way that this pattern may be explained is that the work-related RSI problem in BC was substantially worse, and that the greater work-related RSI burden led to sufﬁcient motivation to pass speciﬁc regulations. Sometimes regulatory initiatives have delayed effects, so a future research question would be to examine more recent work-related RSI trends by province. Another more artifactual explanation is that, through passage of speciﬁc work-related RSI regulations, BC workers as a group were more aware of the issue of work-related RSIs and were, in turn, more willing to attribute RSIs to work causes and/or report work-related RSIs on the survey.



There are several limitations to this study that need to be noted when interpreting the present results. The research design was cross-sectional and observational, so the causal relationships between variables cannot be determined. Another limitation is that the outcome variable was a self- reported measure that was operationalized quite broadly—for example some other studies on gender differences and work- related MSK complaints obtained detail on which site (e.g., neck and shoulder) rather than in the aggregate as in the present study. In addition, neither job title nor a measure of speciﬁc hazard exposures was available in the present health survey. Such potential misspeciﬁcation issues for both the predictors and the outcome variables would tend to reduce the strength of the associations observed. Two strengths of the present study are its representative sample and the use of the same measure across all provinces regarding work- related RSIs.

The implications of these data are that a more tailored, gender speciﬁc approach to prevention is warranted. Also, there appear to be important between-province differences that indicate that monitoring and surveillance of work injury across provinces is an important practical issue for preven- tion, as well as providing an opportunity to compare occupational health and safety initiatives and performance in the aggregate to other provinces. Also, a fruitful area for future research would be the causes of these provincial differences.

## ACKNOWLEDGEMENTS

We would like to acknowledge Albana Çanga for her help in preparing the manuscript and Dr. Karen Messing for her comments on a previous draft. This work was supported by the Canadian Institutes of Health Research grant number #93981. Statistics Canada provided access to the survey data for this study. In terms of competing interests, there are none to declare.

## REFERENCES

Allison P. 1999. Comparing logit and probit coefﬁcients across groups. Sociol Methods Res 28:186–208.

Baruch G, Barnett R. 1985. Role quality, multiple role involvement, and psychological well-being in midlife women. J Pers Soc Psychol 51:578– 585.

Breslin FC, Smith P. 2006. Trial by ﬁre: A multivariate examination of the relation between job tenure and work injuries. Occup Environ Med 63:27–32.

Breslin FC, Smith PM, Dunn JR. 2007. An ecological study of regional variation in work injuries among young workers. BMC Public Health 7:1–12.

Browne WJ. 2009. MCMC estimation in MLwiN Version 2.13. Bristol, UK: Centre for Multilevel Modelling, University of Bristol.

Chung J, Cole D, Clarke J. 2000. Women, work, and injury. In: Sullivan T, editor. Injury and the new world of work. Vancouver: UBC Press, pp. 69–90.

Clayton D, Hills M. 1993. Statistical models in epidemiology. New York: Oxford UP, p. 367.

Coury H, Portcatti I, Alem M, Oishi J. 2002. Inﬂuence of gender on work-related musculoskeletal disorders in repetitive tasks. Int J Ind Ergon 29:33–39.

de Smet P, Sans S, Dramiax M, Boulenguez C, de Backer G, Ferrario M. 2005. Gender and regional differences in perceived job stress across Europe. Eur J Public Health 15:536–545.

de Zwart BC, Broersen JP, Frings-Dresen MH, van Dijk FJ. 1997. Musculoskeletal complaints in The Netherlands in relation to age, gender and physically demanding work. Int Arch Occup Environ Health 70(5):352–360.

European Agency for Safety and Health at Work. 2003. Gender issues in safety and health at work: A review. Luxembourg: Ofﬁce for Ofﬁcial Publications of the. European Communities; http://osha.europa.eu/en/ publications/reports/209 Accessed June 13, 2012.

Fielding A, Goldstein H. Cross-classiﬁed and multiple membership structures in multilevel models: An introduction and review. Birming- ham, UK: University of Birmingham, ISBN 1 84478 797 2.2006.

Hagberg M, Wegman DH. 1987. Prevalence rates and odds ratios of shoulder-neck diseases in different occupational groups. Br J Ind Med 44:602–610.

Hooftman WE, van Poppel MN, van der Beek AJ, Bongers PM, van Mechelen W. 2004. Gender differences in the relations between work- related physical and psychosocial risk factors and musculoskeletal complaints. Scand J Work Environ Health 30(4):261–278.

Hooftman WE, van der Beek AJ, Bongers PM, van Mechelen W. 2009. Is there a gender difference in the effect of work-related physical and psychosocial risk factors on musculoskeletal symptoms and related sickness absence? Scand J Work Environ Health 35(2):85–95.

Janwantanakul P, Pensri P, Jiamjarasrangsri V, Sinsonqsook T. 2008. Prevalence of self-reported musculoskeletal symptoms among ofﬁce workers. Occup Med (Lond) 58(6):436–438.

Karlqvist L, Tornqvist E, Hagberberg M, Hagman M, Toomingas A. 2002. Self-reported working conditions of VDU operators and associations with musculoskeletal symptoms: A cross-sectional study focusing on gender differences. Int J Ind Ergon 30:277–294.

Krause N, Ragland DR, Greiner BA, Fisher JM, Holman DL, Selvin S. 1997. Physical workload and ergonomic factors associated with prevalence of back and neck pain in urban transit operators. Spine 22 (18):2117–2126.

Labour Force Survey. 2012. Labour force characteristics by sex and age group. Ottawa, Ontario: Statistics Canada. http://www.statcan.gc.ca/ tables-tableaux/sum-som/l01/cst01/labor05-eng.htm?sdi¼women Ac- cessed May 14, 2012.

Lawson AB, Browne WJ, Vidal Rodeiro CL. 2003. Disease mapping with WINBUGS and MLwiN. Chichester, UK: Wiley.

Leino-Arjas P. 1998. Smoking and musculoskeletal disorders in the metal industry: A prospective study. Occup Environ Med 55(12):828– 833.

Lohr S. 1999. Sampling: design and analysis. Toronto: Duxbury Press.

McDonough P. 1997. The social patterning of work disability among women in Canada. J Disabil Policy Stud 8:75–95.

Messing K, Stellman J. 2006. Sex, gender, and women’s occupational health: The importance of considering mechanism. Environ Res 101:149–162.

Messing K, Tissot F, Stock S. 2008. Distal lower-extremity pain and work postures in the Quebec population. Am J Public Health 98(4):705– 713.

Messing K, Tissot F, Stock SR. 2009. Should studies of risk factors for musculoskeletal disorders be stratiﬁed by gender? Lessons from the 1998 Quebec Health and Social Survey. Scand J Work Environ Health 35(2):96–112.

Mustard CA, Etches J. 2003. Gender differences in socioeconomic inequality in mortality. J Epidemiol Community Health 57(12):974– 980.

Mustard CA, Frohlich N. 1995. Socioeconomic status and the health of the population. Med Care 33(12 Suppl):DS43–DS54.

Rantanen J, Kinnunen U, Mauno S, Tillemann K. 2011. Introducing theoretical approaches towork-life balance and testing a new typology among professionals. In: Kaiser S, Ringlstetter MJ, Eikhof DR, Pina e Cunha M, editors. Creating Balance? International Perspectives on the Work-Life Integration of Professionals. Berlin: Springer, pp. 27–46.

Rasbash J, Browne W, Goldstein H, Yang M, Plewis I, Healy M, Woodhouse M, Draper G, Langford D, Lewis T. 2000. A user’s guide to MLwiN. Bristol, England: University of Bristol.

Reid J, Hardy M. 1999. Multiple roles and well-being among midlife women: Testing role strain and role enhancement theories. J Gerontol 54B:S329–S338.

Roberts G. 2000. Combining data from the ﬁrst two Panels of SLID for carrying out longitudinal analysis. Unpublished Report. Ottawa, Ontario: Statistics Canada.

Sandmark H. 2009. Job mismatching, unequal opportunities and long- term sickness absence in female white-collar workers in Sweden. Scand J Public Health 37(1):43–49.

Silverstein B, Fine L, Stetson D. 1987. Hand-wrist disorders among investment casting plant workers. J Hand Surg 12A(5 Pt 2):838–844.

Smith GD, Hart C, Watt G, Hole D, Hawthorne V. 1998. Individual social class, area-based deprivation, cardiovascular disease risk factors, and mortality: The Renfrew and Paisley Study. J Epidemiol Community Health 52(6):399–405.

Statistics Canada. 1992. Standard geographical classiﬁcation 1991— The classiﬁcation, Volume 1. Ottawa: Statistics Canada, pp. 153–160.

Statistics Canada. 2003a. Canadian Community Health Survey: User Guide for the Public Use Microdata File. Ottawa,Ontario: Ministry of Industry,82M0013GPE.http://www23.statcan.gc.ca:81/imdb-bmdi/ pub/document/3226_D7_T9_V2-eng.pdf Accessed June 13, 2012.

Statistics Canada. 2003b. Canadian Census for 2001 Dictionary. Catalogue92-378-XIE.http://publications.gc.ca/Collection/Statcan/ 92-378-X/92-378-XIE02002.pdf Accessed June 13, 2012.

Statistics Canada. 2005. Canadian Community Health Survey, Cycle 3.1, Final questionnaire. Ottawa, Ontario: Statistics Canada.

Statistics Canada. 2006a. Canadian Community Health Survey Cycle 3.1 (2005). Public use microdata ﬁle user guide. Ottawa, Ontario: Statistics Canada. http://www23.statcan.gc.ca:81/imdb-bmdi/pub/doc- ument/3226_D7_T9_V3-eng.pdf Accessed June 13, 2012.



Statistics Canada. 2006b. Canadian Community Health Survey, Cycle 3.1, derived variable speciﬁcation. Ottawa, Ontario: Statistics Canada.

Statistics Canada. 2010. Retail sector activity. In: Statistics Canada. State of Retail: The Canadian Report 2010. Ottawa, Ontario: Statistics Canada.http://www.ic.gc.ca/eic/site/retra-comde.,nsf/eng/qn00285. html Statistics Canada. 2011. Labour: Unemployment rate by selected age groups. Ottawa, Ontario: Statistics Canada. http://www.statcan.gc.ca/ pub/11-402-x/2010000/chap/lab-tra/lab-tra-eng.htm,2011Accessed May 14, 2012.



Sullivan T. 2000. Injury and the new world of work. Vancouver: UBC Press, p. 368.

Tucker E. 2003. Diverging trends in worker health and safety protection and participation in Canada, 1985–2000. Ind Relat 58(3):395–423.
sought dental care without signiﬁcant oral changes and with a history of regular visits to the dentist. His main complaints were constipation and anal/rectal bleeding upon defecation. Mucocu- taneous pigmentations were observed in the perioral, palmar, and plantar regions. Abdominal X-ray with contrast, endoscopy, and excisional biopsy were performed, and the ﬁndings conﬁrmed the diagnosis of PeutzeJeghers syndrome.

PCC-196 - PLEOMORPHIC ADENOMA OF THE HARD PALATE:CASEREPORT. FRANCIELLESANTOS NASCIMENTO, RAFAELLE RODRIGUES CAVALCANTI, FERNANDA BRAGA PEIXOTO, SONIA MARIA SOARES FERREIRA, JOSÉ DE AMORIM LISBOA NETO, RICARDO VIANABESSANOGUEIRA,CAMILAMARIABEDER RIBEIRO. CENTRO UNIVERSITÁRIO CESMAC.

Pleomorphic adenoma (PA) is the most common benign sali- vary gland neoplasm, primarily affecting the parotid gland. It af- fects individuals within a wide age range, with a slight predilection for females. Conservative enucleation is the suggested surgical approach, and the risk of malignancy is low  $(5\%)$ . A 57-year-old female presented with a single, asymptomatic, normochromic-to- violaceous tumor, in the hard palate region, that had ﬁrst appeared 2 years earlier. The diagnostic hypotheses were PA, adenoid cystic carcinoma, and polymorphous low-grade adenocarcinoma. A bi- opsy was performed, and histological examination of the sections suggested a PA composed of myoepithelial and ductal cells, with a ﬁbrous capsule, and myxoid stroma in some areas. The tumor was surgically removed, and patient remains under followed-up. This case shows that PA can occur in the minor salivary gland and emphasizes its differential diagnosis with other tumors.

PCC-197 - USE OF CARNOY’S SOLUTION AS A CON- SERVATIVE ALTERNATIVE IN THE TREATMENT OF AGGRESSIVE ODONTOGENIC LESIONS: A SERIES OF 12 CASES. MÁRCIO MENEZES NOVAES, JOSÉ SANDRO PEREIRA DA SILVA, PETRUS PEREIRA GOMES, LÉLIA BATISTA DE SOUZA, VICTOR DINIZ BORBOREMA DOS SANTOS,HAROLDOABUANAOSÓRIOJUNIOR, ADRIANO ROCHA GERMANO. UNIVERSIDADE FED- ERAL DO RIO GRANDE DO NORTE.

The treatment of the bone surface after enucleation of odon- togenic cysts and tumors, aimed at eliminating the remnants of a lesion, has proven to be an effective therapy, avoiding or delaying bone resection. Among the various forms of surface treatment, the use of Carnoy’s solution is widespread. This retrospective study shows the experience of a surgical service in the treatment of aggressive odontogenic lesions between 2009 and 2013. We evaluated 12 cases: 4 of ameloblastoma; 7 of keratocystic odon- togenic tumor; and 1 of odontogenic cyst. All cases were treated by enucleation with peripheral ostectomy, followed by the application of Carnoy’s solution for  $5\ \mathrm{min}$ . The follow-up ranged from 8 to 48 months. We observed a high success rate and low morbidity, demonstrating the effectiveness of this conservative therapy.

PCC-198-EXTRAFOLLICULARADENOMATOID ODONTOGENIC TUMOR: REPORT OF AN UNCOM- MON CASE. FERNANDA DA SILVA BARROS, POLYANNA ATAIDE SOARES, JANAINA ANDRADE LIMA SALMOS DE BRITO, VANIO SANTOS COSTA, LUIZ ARTHUR BARBOSA DA SILVA, RICARDO VIANA BESSA NOGUEIRA, JOSÉ DE AMORIM LISBOA NETO. UNIVERSIDADE FEDERAL DE ALAGOAS.

Adenomatoid odontogenic tumor (AOT) is a benign odon- togenic tumor of epithelial origin, with slow but progressive growth. This tumor has 3 variants: follicular  $(73\%)$ , extra- follicular  $(24\%)$ , and peripheral  $(3\%)$ . It is commonly located in the upper jaw of young patients and has a predilection for females (2:1). A 77-year-old Black male complained of a swelling in the anterior mandible. Intraoral examination revealed expansion of cortical bone and displacement of the anterior teeth. Panoramic radiography showed a well-deﬁned radiolucent lesion, extending from tooth #33 to tooth #43, with resorption of the apical portions of the roots of the teeth involved and no disruption of cortical bone. An incisional biopsy was performed, and histological examination revealed odontogenic epithelial neoplasm composed of spindle cells, duct-like structures, and acellular calciﬁcations. The deﬁnitive diagnosis was extra- follicular AOT.

PCC-199 - ESTIMATED TIME FOR THE DEVELOPMENT OF AN AMELOBLASTOMA: A CASE REPORT. RENAN PIRESGONZAGA,THAMIRYSDANTASNÓBREGA, KEZIARAPHAELADELUCENAALVES,ROBÉRIA LÚCIA DE QUEIROZ FIGUEIREDO, GUSTAVO JOSÉ DE LUNA CAMPOS, POLLIANNAMUNIZ ALVES, TONY SANTOSPEIXOTO.UNIVERSIDADEESTADUALDA PARAÍBA.

Ameloblastoma is an odontogenic tumor of epithelial origin that exhibits local inﬁltrative and aggressive behavior, accounting for  $1\%$  of all odontogenic tumors. The posterior region of the mandible is the most commonly affected site for this neoplasm. Due to a lack of symptoms, some cases of amelo- blastoma are diagnosed during routine radiographic examina- tions. The objective of this study was to report the case of a 29-year-old White patient who presented with a unicystic ame- loblastoma in the anterior region of the mandible that was not detected during 4 years of orthodontic treatment. The elapsed time for the development of unicystic ameloblastoma, the ther- apeutic approach for the present case, and the importance of periodic radiographic follow-up of patients under orthodontic treatment are discussed.

PCC-200 - AMELOBLASTIC FIBRO-ODONTOMA: A CASE REPORT. KARLA SANTOS PINTO, GABRYELLE NUNES SANTOS, JOSÉ ALBERTO RABELO DE JESUS JÚNIOR, RAISSA MELO HENRIQUES, RICARDO LUIZ CAVALCANTIDEALBUQUERQUEJÚNIOR,PAULO ALMEIDA JÚNIOR. UNIVERSIDADE TIRADENTES.

Ameloblastic ﬁbro-odontoma (AFO) is a benign odontogenic tumor consisting of odontogenic ectomesenchyme and, epithelial strands/nests in conjunction with the presence of dentin and enamel. A 14-year-old female presented a painful vestibular swelling in the right molars region. Imaging revealed multiple, small well-circumscribed bone cavities surrounded by an osteo- lytic area, which was in turn surrounded by a thin sclerotic line. An incisional biopsy revealed a cell-rich myxoid connective tis- sue, with strands and nests of odontogenic epithelium, in asso- ciation with foci of dentin-like and enamel-like material. The diagnosis was AFO. A surgical enucleation was performed, and there had been no recurrence after 2 years of follow-up. An AFO is a rare mixed odontogenic tumor with similarities to amelo- blastic ﬁbroma and ameloblastic dentinoma. The nature of and relationships among mixed odontogenic tumors and related lesions are still controversial.
sought dental care without signiﬁcant oral changes and with a history of regular visits to the dentist. His main complaints were constipation and anal/rectal bleeding upon defecation. Mucocu- taneous pigmentations were observed in the perioral, palmar, and plantar regions. Abdominal X-ray with contrast, endoscopy, and excisional biopsy were performed, and the ﬁndings conﬁrmed the diagnosis of PeutzeJeghers syndrome.

PCC-196 - PLEOMORPHIC ADENOMA OF THE HARD PALATE:CASEREPORT. FRANCIELLESANTOS NASCIMENTO, RAFAELLE RODRIGUES CAVALCANTI, FERNANDA BRAGA PEIXOTO, SONIA MARIA SOARES FERREIRA, JOSÉ DE AMORIM LISBOA NETO, RICARDO VIANABESSANOGUEIRA,CAMILAMARIABEDER RIBEIRO. CENTRO UNIVERSITÁRIO CESMAC.

Pleomorphic adenoma (PA) is the most common benign sali- vary gland neoplasm, primarily affecting the parotid gland. It af- fects individuals within a wide age range, with a slight predilection for females. Conservative enucleation is the suggested surgical approach, and the risk of malignancy is low  $(5\%)$ . A 57-year-old female presented with a single, asymptomatic, normochromic-to- violaceous tumor, in the hard palate region, that had ﬁrst appeared 2 years earlier. The diagnostic hypotheses were PA, adenoid cystic carcinoma, and polymorphous low-grade adenocarcinoma. A bi- opsy was performed, and histological examination of the sections suggested a PA composed of myoepithelial and ductal cells, with a ﬁbrous capsule, and myxoid stroma in some areas. The tumor was surgically removed, and patient remains under followed-up. This case shows that PA can occur in the minor salivary gland and emphasizes its differential diagnosis with other tumors.

PCC-197 - USE OF CARNOY’S SOLUTION AS A CON- SERVATIVE ALTERNATIVE IN THE TREATMENT OF AGGRESSIVE ODONTOGENIC LESIONS: A SERIES OF 12 CASES. MÁRCIO MENEZES NOVAES, JOSÉ SANDRO PEREIRA DA SILVA, PETRUS PEREIRA GOMES, LÉLIA BATISTA DE SOUZA, VICTOR DINIZ BORBOREMA DOS SANTOS,HAROLDOABUANAOSÓRIOJUNIOR, ADRIANO ROCHA GERMANO. UNIVERSIDADE FED- ERAL DO RIO GRANDE DO NORTE.

The treatment of the bone surface after enucleation of odon- togenic cysts and tumors, aimed at eliminating the remnants of a lesion, has proven to be an effective therapy, avoiding or delaying bone resection. Among the various forms of surface treatment, the use of Carnoy’s solution is widespread. This retrospective study shows the experience of a surgical service in the treatment of aggressive odontogenic lesions between 2009 and 2013. We evaluated 12 cases: 4 of ameloblastoma; 7 of keratocystic odon- togenic tumor; and 1 of odontogenic cyst. All cases were treated by enucleation with peripheral ostectomy, followed by the application of Carnoy’s solution for  $5\ \mathrm{min}$ . The follow-up ranged from 8 to 48 months. We observed a high success rate and low morbidity, demonstrating the effectiveness of this conservative therapy.

PCC-198-EXTRAFOLLICULARADENOMATOID ODONTOGENIC TUMOR: REPORT OF AN UNCOM- MON CASE. FERNANDA DA SILVA BARROS, POLYANNA ATAIDE SOARES, JANAINA ANDRADE LIMA SALMOS DE BRITO, VANIO SANTOS COSTA, LUIZ ARTHUR BARBOSA DA SILVA, RICARDO VIANA BESSA NOGUEIRA, JOSÉ DE AMORIM LISBOA NETO. UNIVERSIDADE FEDERAL DE ALAGOAS.

Adenomatoid odontogenic tumor (AOT) is a benign odon- togenic tumor of epithelial origin, with slow but progressive growth. This tumor has 3 variants: follicular  $(73\%)$ , extra- follicular  $(24\%)$ , and peripheral  $(3\%)$ . It is commonly located in the upper jaw of young patients and has a predilection for females (2:1). A 77-year-old Black male complained of a swelling in the anterior mandible. Intraoral examination revealed expansion of cortical bone and displacement of the anterior teeth. Panoramic radiography showed a well-deﬁned radiolucent lesion, extending from tooth #33 to tooth #43, with resorption of the apical portions of the roots of the teeth involved and no disruption of cortical bone. An incisional biopsy was performed, and histological examination revealed odontogenic epithelial neoplasm composed of spindle cells, duct-like structures, and acellular calciﬁcations. The deﬁnitive diagnosis was extra- follicular AOT.

PCC-199 - ESTIMATED TIME FOR THE DEVELOPMENT OF AN AMELOBLASTOMA: A CASE REPORT. RENAN PIRESGONZAGA,THAMIRYSDANTASNÓBREGA, KEZIARAPHAELADELUCENAALVES,ROBÉRIA LÚCIA DE QUEIROZ FIGUEIREDO, GUSTAVO JOSÉ DE LUNA CAMPOS, POLLIANNAMUNIZ ALVES, TONY SANTOSPEIXOTO.UNIVERSIDADEESTADUALDA PARAÍBA.

Ameloblastoma is an odontogenic tumor of epithelial origin that exhibits local inﬁltrative and aggressive behavior, accounting for  $1\%$  of all odontogenic tumors. The posterior region of the mandible is the most commonly affected site for this neoplasm. Due to a lack of symptoms, some cases of amelo- blastoma are diagnosed during routine radiographic examina- tions. The objective of this study was to report the case of a 29-year-old White patient who presented with a unicystic ame- loblastoma in the anterior region of the mandible that was not detected during 4 years of orthodontic treatment. The elapsed time for the development of unicystic ameloblastoma, the ther- apeutic approach for the present case, and the importance of periodic radiographic follow-up of patients under orthodontic treatment are discussed.

PCC-200 - AMELOBLASTIC FIBRO-ODONTOMA: A CASE REPORT. KARLA SANTOS PINTO, GABRYELLE NUNES SANTOS, JOSÉ ALBERTO RABELO DE JESUS JÚNIOR, RAISSA MELO HENRIQUES, RICARDO LUIZ CAVALCANTIDEALBUQUERQUEJÚNIOR,PAULO ALMEIDA JÚNIOR. UNIVERSIDADE TIRADENTES.

Ameloblastic ﬁbro-odontoma (AFO) is a benign odontogenic tumor consisting of odontogenic ectomesenchyme and, epithelial strands/nests in conjunction with the presence of dentin and enamel. A 14-year-old female presented a painful vestibular swelling in the right molars region. Imaging revealed multiple, small well-circumscribed bone cavities surrounded by an osteo- lytic area, which was in turn surrounded by a thin sclerotic line. An incisional biopsy revealed a cell-rich myxoid connective tis- sue, with strands and nests of odontogenic epithelium, in asso- ciation with foci of dentin-like and enamel-like material. The diagnosis was AFO. A surgical enucleation was performed, and there had been no recurrence after 2 years of follow-up. An AFO is a rare mixed odontogenic tumor with similarities to amelo- blastic ﬁbroma and ameloblastic dentinoma. The nature of and relationships among mixed odontogenic tumors and related lesions are still controversial.
