Chaos Rules!
Time-series analysis (TSA) is one of the primary statistical techniques used in determining behavioral characteristics of time-sequential data. TSA has been applied for many years in mining such data for trends, forecasting information, intervention analysis (did something cause a change in the data?), cross correlation between data sets, seasonal variations, etc. TSA is employed in numerous fields, such as politics, social sciences, medicine, biostatistics, and engineering. Here, however, its most important use is in helping to reveal the presence of chaos in experiments.
Using some of TSA's tricks, attractors buried within experimental data can be uncovered. Whether the hidden attractor turns out to be strange or just periodic, applying TSA techniques, such as lagging, varying embedding dimensions, determining correlation dimensions, etc., will help us in separating random phenomena from the truly chaotic. Recreated attractors based on single-variable data are plotted in a pseudo phase space as will be seen below. The term pseudo (or lagged) phase space distinguishes our plot from one representing the state of the original dynamic system plotted in real phase space with all the system's many variables.
As discussed previously, an attractor is the phase space set of points that represents the equilibrium state or states of a dynamic system. For the perpetually swinging (forced) pendulum, it's an ellipse or a circle. For a damped pendulum, it's a fixed point. But for chaotic systems, it really gets interesting.
Takens' Theorem
In 1981, Floris Takens published a paper with a remarkable conclusion regarding obtaining dynamical systems' states from limited data. Please note that what follows is a highly simplistic translation of Takens' rigorous explanation given in his now classic paper. Time-sequential data obtained from a single variable recorded from a dynamic, multidimensional system can be used to reproduce an attractor in pseudo phase space. That attractor is a valid representation of the attractor that exists in the real phase space of the original dynamic system. For that validity to be guaranteed, the data must be smooth (so that derivatives can be obtained), cross-coupled in some fashion (so that the data properly represents the entire dynamic system), and properly embedded. Proper embedding is the tricky part. What this means for us is that we can use a one-dimensional table of x values to reproduce an attractor that is valid and not just a pretty picture. This is like using a table of sequentially measured temperatures to recreate an attractor of the entire atmospheric weather pattern in a region! The guarantee for validity assumes that the smooth (differentiable) recorded temperature values are coupled in some way to barometric pressure, wind speed, humidity, etc., and are properly embedded. (The fact that the word guarantee is used means that the single-variable attractor could still be valid even if the theorem's requirements aren't met. It's just that you won't be able to complain to Floris Takens if the attractor is bogus.)
The first experiments on this page were performed solely in software. Then, a few computer-assisted laboratory experiments known to produce chaos under the right conditions were carried out.
Limit cycles (periodic attractors)
A reasonable way to explain the nature of attractors is to demonstrate several examples by creating them on a computer. The following Mathematica command generated the data file that was used to produce both the standard, two-dimensional x-y plot and the animated, lagged time-series plot seen at right (also plotted in 2D):
plotTable = Table[ Sin[x]3 , {x, 0, 10Pi, .01 Pi}];
The data in the plotTable file are just the values for plotting points on the ordinate or y-axis. Mathematica assumes equally spaced x-values (e.g., 0,1,2,3...) for a simple x-y plot.
To use time-series techniques for the animated plot, my program interpreted the plotTable file as a sequential series of x- and y-values with specified point lags. The program then stored the revamped data in a new file. (In Mathematica, a ListPlot command was used to conveniently plot the reanalyzed file.) To show how lags are used, let's assume that our revamped data file contains only nine numbers in the following order:
1,2,3,4,5,6,7,8,9
To plot these values with a lag of 1, the eight plottable x-y points are
(1,2),(2,3),(3,4),(4,5),(5,6),(6,7),(7,8),(8,9)
For a lag = 2, the seven plottable points are
(1,3),(2,4),(3,5),(4,6),(5,7),(6,8),(7,9)
And for a lag = 5, the four plottable points are
(1,6),(2,7),(3,8),(4,9)
The animated plots seen here and in the next example were plotted for lags ranging from 1 to 100. It's easy to see that large data files are needed if extensive lags are of interest since the number of points that can be plotted decreases with an increase in lag value. Finally, the lagged plot shown at right reveals a limit cycle or periodic attractor. All calculated points travel the closed-loop attractor forever.
Fixed-point attractors
At right is an example of a fixed-point attractor plotted in 2D. The data for the plot was generated with this command:
plotTable = Table[ e-0.1x Sin[x] , {x, 0, 10Pi, .01 Pi}];
The x-y plot shows the function executing an exponential decay. This emulates a damped pendulum slowly grinding to a halt at the center of its swing.The phase-space images show a contracting circle spiraling clockwise inwardly to a fixed-point attractor at its center.
Had I used a positive exponent for e rather than a negative one, the x-y plot would have shown an exponentially increasing waveform. The animated plot would have shown a spiral expanding clockwise. The plot would have been the mirror image of the one seen here. And, the fixed-point attractor would have been located at infinity!
As in the previous lagged plot, this plot was drawn with two axes (embedding dimension = 2) and the resulting animated images are shown in phase space. The images are analogous to Lissajous figures in which signals differing only in phase are plotted one against the other.
Fixed-point attractors are the simplest kind of attractor. The fixed point may represent an object in constant motion or one which has completely stopped, as in the example. Whether motion or some other activity, such as respiration, is being examined, the final equilibrium state can be represented by a solitary fixed point in phase space. And the rest is trajectory.
I emphasized the two-dimensional aspect for the lag plots seen above. If, instead of using pairs of points I plotted triplets on three axes, a 3D plot would be generated, of course. (Given a sufficiently powerful computer, there is no limit to the number of axes that can be used to evaluate a lagged time series. However, in order to display the results, only a suitable 2D projection of the multidimensional solution can be used.)
For example, let's again use our very small data file containing only the following points:
1,2,3,4,5,6,7,8,9
If we want to plot these values in three dimensions with, say, a lag of 2, the five plottable x-y-z points are
(1,3,5),(2,4,6),(3,5,7),(4,6,8),(5,7,9)
When triplets of data points are used to generate a 3D graph, regardless of the lag value used, the embedding dimension is three. The 3D technique will be used to plot some strange attractors to follow.
Strange attractors
The term was coined by David Ruelle and Floris Takens. Strange attractors are the attractors of chaotic systems and often possess fractal properties.
In addition to Edward Lorenz's attractor that was previously shown, another one of the most recognizable attractors was developed by the mathematician and astronomer, Michel Hénon. Unlike Lorenz's differential equations for fluid convection, the two Hénon equations do not represent a particular system. The variables, however, are coupled and thus fulfill one of Takens' requirements. Here are the equations and commands I used to generate the complete dynamic system's data in Mathematica:
x[0] = 1.5; y[0] = 1.5;
x[n_] := x[n] = y[n - 1] + 1 - 1.4 (x[n - 1])2;
y[n_] := y[n] = 0.3x[n - 1];
plotXtable = Table[{x[n], y[n]}, {n, 1, 1000}];
The top plot at right is a graph of the two equations (and variables) comprising the Hénon attractor. As such, it represents the entire dynamic system. That is, the x and y values are the numbers calculated with the equations above. However, the second plot was created from only the x values. I experimented with different lags, and the plot shows the results of lag values ranging from 1 to 10. Only for a lag value of 1 does the plot resemble the original dynamic system. Plots with a lag of 2 and above bear little resemblance to the original. Soon the plots appear random.
A 2D plot only requires an x value (I could have chosen the y values just as well) and the same variable with a lag. A 3D plot requires three values. If one of these is the variable xn , the two additional values of that same variable but obtained with different time lags would be xn+2 and xn+3. In this example, generating a 3D plot would not be useful.
The attractor's sound was generated from the x-variable file that was extracted from the system.
For the Hénon attractor, a 2D plot with a lag of 1 is sufficient. The embedding dimension is 2.
Drip...drip...drip...drip....
Few things are more annoying than lying in bed on a cold, wintry night listening to the monotonous cadence of a dripping faucet nearby. Annoying, that is, unless you're a physicist, in which case the dripping sound is probably accompanied by visions of period-doubling bifurcations, limit cycles, and chaos all dancing in your head.
Back in the 1970s Robert Shaw, working in collaboration with Peter Scott (and others) when they were at the University of California, Santa Cruz, decided to experiment with a laboratory mockup of a dripping faucet. Shaw set up an apparatus to precisely control the flow of water so that dropping patterns ranging from a steady drip to a smoothly flowing stream could be produced. Shaw monitored the falling drops as they intercepted a laser beam. The detected interruptions were sensed and recorded with electronic instrumentation, which included an oscilloscope. As the flow was slowly increased and the measurements proceeded, Shaw detected dropping patterns ranging from equally spaced drops through a regime of period doubling (bifurcations) and on to chaotic flow. As points were collected, Shaw decided to try plotting the dropping water using lags between one drop and the next. Sure enough, obvious attractors appeared. Now, this is an experiment I wanted to replicate—both for the experience and to sharpen some of the tools I intend to use for future chaos hunting.
My setup
As seen at upper right, the source bottle acts as the primary supply of water for the dripping nozzle. The reservoir, physically located higher up, replaces water drawn from the source bottle as rapidly as it is drained. The source bottle siphons water out of the reservoir with suction. The replacement drops fall through space onto the surface of the water remaining in the source bottle. Since the idea is to maintain a constant head (pressure) in order to prevent the water flow from diminishing over time, the tube from the reservoir must not directly contact the water in the source bottle.
The aspirator-type source bottle feeds a rotameter (which I previously used for controlling gas flow) that allows fine control of water flow. The drops that leave the nozzle intercept a laser beam that is directed at a photodiode. The light sensor feeds its output signal to the following devices:
(1) A storage scope that permits scrutinizing the dropping water for periods and chaos (a view is shown at lower right).
(2) A speaker to allow using the sound of drops as a further indicator of periodic or chaotic behavior.
(3) A period meter to determine the time between drops and a drop counter. The drop counter can be operated in frequency mode to enable the drops/second rate to be viewed.
(4) A stroboscope that triggers whenever a drop falls, except when the computer is online taking data. I use this instrument to study the behavior of drops in periodic mode and in chaos. Also, photos and videos of the dripping water are produced with the aid of the device.
There is also a partially reflective beam splitter + photogate pair connected to a computer interface (not shown in the drawing). The beam splitter is interposed in the laser beam path shown. To perform the experiment, I adjust the rotameter while simultaneously monitoring the falling drops on the oscilloscope. If "drops of interest" do appear, I can quickly trigger the computer to begin collecting data.
Finally, the drops are funneled into a drain bottle and manually returned to the reservoir.
Whence the nonlinearity?
There have been numerous books written about chaos. Many years ago, Nova even aired a documentary about the subject. Therefore, it was difficult not to have been aware of a relationship between a dripping faucet and the onset of something called chaos. What that relationship was, I never really knew. But, in actively dealing with nonlinear phenomena exhibiting chaotic behavior, one must ask the following question: What is the source of the observed nonlinearity? The flow of water exiting the source bottle and passing through capillary channels—the transport tubing and rotameter—obeys Poiseuille's Law. Accordingly, the apparatus can be arranged so that the flow is linearly related to the source pressure and capillary dimensions. The flow is laminar with no hint of any turbulence, and yet the ability to produce chaos persists. So, the nonlinearity does not originate there.
Is the source the liquid's temperature or viscosity? In an experiment such as this, it is not overly difficult to come up with alternative schemes that can be made to work after a fashion. For example, instead of droplets of liquid descending in a gas, bubbles of gas rising in a liquid can be tried. Some researchers maintain that these two are really just inverse techniques while other researchers have performed experiments to dispel that belief. Experiments employing drops of lighter liquids rising in denser liquids or drops of denser liquids descending in lighter liquids are doable. Viscosities and temperatures can be altered. Chaos can probably be generated within all these variations (I haven't tried it), but the dynamic mechanisms are likely to be quite different. At any rate, in this experiment the source of the nonlinearity lies elsewhere.
A closer look at drop formation near the tip of the nozzle will supply a needed clue. To do this without benefit of a high-speed camera, I used the stroboscope.
(The four images shown at right in this animation were taken of four different drops using a variable delay circuit as part of the stroboscope-tripping mechanism. The variable delay allows me to tune to a specific point in time of the falling drop process that I want to photograph.)
These slowly forming drops were falling within a periodic region. After pinching off, the largest part of the drop falls, and there is a snapback of the water droplet still adhering to the glass tip. When the drops are arriving at the nozzle fast and furiously, the snapback does not fully complete as it does here. Shaw (1984) modeled the drop using an analog computer and demonstrated how the drop acts like a weight on a spring. In this case, the restraining force of the spring simulates surface tension. As the drop falls, the water column that suspended it rebounds and oscillates. If the rate of drop fall is increased into the chaos regime, each falling drop nonlinearly modifies the timing of the next drop due in part to oscillations of the water column. Also, the timing of the falling drop affects the time needed to reform the next drop before it falls. There is a feedback action taking place. The same scenario applies recursively to the following drops as well. Hence the nonlinearity.
The experiments
Below are some of the results derived from computerized data collection for two dropping water experiments. The first experiment recorded drops operating in a period 2 mode. That is, every other drop had equal time spacings (e.g., 1 sec, 3 sec, 1 sec, 3 sec, 1 sec, 3 sec, etc.) instead of every drop as would be the case for period 1. Drops in the second experiment, flowing at a rate of approximately 15 drops per second, operated in the chaotic regime.
Note: The experiments' data were collected using a LabPro interface from Vernier. It is an inexpensive device with controlling software that is useful for dropping-water experiments. LabPro, coupled with Vernier's photogate sensor, is usually fast enough to collect data in these experiments. The collected data was stored in a file that was subsequently read in by Mathematica for calculations and plotting. In the absence of a commercial interface, an IR LED paired with a suitable photodiode can be constructed to act as a photogate. The output of that pairing can be read in through the PC's parallel port with a high-speed software routine written in C or assembly language.
At the extreme right is a histogram of the original data. Next over is an FFT. The more "grass" visible on the FFT, the greater the chaos content with respect to any periodicity. To the left of that are two images of the attractors. (To me, the chaotic attractor sort of resembles a flying camel. Such similarities to animals has led some researchers to refer to these attractors as a zoo!) Finally, below the rotating 3D attractors are the audio controls for the attractors.
The VRAs for the two experiments are seen at right. They are displayed greatly magnified to reveal any hidden patterns indicative of chaos. The VRA image at the extreme right is that of chaotically dropping water. Note the slanting lines that signify chaos. The VRA of the periodic drops seen at near right does not show the telltale slanting. However, in order to determine their actual periodicity, I relied on the histogram shown above.
When setting up the VRA, the software (see References) enables one to run an analysis on the data file. This, in turn, helps determine the desirable embedding dimension and the average mutual information that allows creating a meaningful plot, as shown.
So, the next time you encounter a recalcitrant faucet, instead of just getting upset consider the forces acting on that drop of water momentarily perched at the spout. There are cohesive, intermolecular forces responsible for the water's surface tension constantly grappling with gravity. Increase the water flow and at some point nonlinear changes take place heralding the onset of chaos. Take the time to ponder how seemingly independent drops form repeatable and recognizable patterns in phase space, almost as if they possess an inorganic intelligence. And then—call the plumber!
Next, a surprising source of chaos. But, it's only visible in the dark!
