In physics, entropy, symbolized by S, is a measure of a system’s degree of organization and ability to do work. At the microscopic level, entropy is a thermodynamic property that measures the degree of randomization or disorder. At the macroscopic level, the production of entropy is associated with a is a loss of ability to do useful work. Energy is degraded to a less useful form, and it is sometimes said that there is a decrease in the availability of energy. The idea that entropy can be produced but never destroyed is one way to state the second law of thermodynamics.

German physicist Rudolf ClausiusThe concept of entropy was developed in the 1850s by German physicist Rudolf Clausius, one of the founders of the science of thermodynamics. Clausius formalized Carnot’s heat engine work, first stated the basic ideas behind the second law of thermodynamics, and in 1865 he introduced the concept of entropy.
 

Macroscopic viewpoint (classical thermodynamics)

One perspective of entropy comes from the field of classical thermodynamics, a branch of physics developed in the nineteenth century, that studied heat and work and their relation to the collision and interaction of particles in large, near-equilibrium systems. The driving forces behind the development of the field were Sadi Carnot, Rudolph Clausius, Benoit Claperyon, James Clerk Maxwell, and William Thomson (Lord Kelvin).

The fundamental, which will be developed in more detail below is stated simply: for an isolated system, its entropy S can never decrease. This statement above tells us that all physical processes in which heat is transformed into work and visa versa must satisfy the condition that entropy must not decrease. In other words, if this condition is not satisfied, the process cannot take place.

Consider a system in contact with a heat source and held at constant temperature. In this case, Clausius defined entropy as:

S = Q /T                 (1)

where S is the entropy, Q is the heat content of the system, and T is the temperature of the system.

For a very small change in the independent parameters of the system such as volume, in which ΔQ amount of heat flows into it via a reversible process, the change in entropy of the system is given by:

ΔS = ΔQ /T             (2)

Clausius, Carnot and others in the classical school were interested in the ability to convert mechanical work into heat energy, and vice versa. Thus, entropy in classical thermodynamics is interpreted simply as a state function of a thermodynamic system; that is, a property depending only on the current state of the system, independent of how that state came to be achieved. The state function has the important property that, when multiplied by a reference temperature, it can be understood as a measure of the amount of energy in a physical system that cannot be used to do thermodynamic work; i.e., work mediated by thermal energy.

This leads to another formulation of equation (2). If you pump energy, i, into a system, what happens to it? Part of the energy goes into the internal heat content, Q, making Q a positive quantity. Some of that energy is also expressed as an amount of mechanical work done by the system (W) An example of W is a hot gas pushing against a piston in a car engine. Another portion of the energy does not participate in the work, and goes into the heat reservoir as ΔQ. So a simple substitution allows equation 2 to be re-written as:

ΔS = (ΔU - ΔW)/T    (3)

This alternate form of the equation also works for heat taken out of a system (ΔU is negative) or work done on a system (W is negative). This provides and additional interpretation of idea of the classical relation between work, energy and entropy.

Entropy and Physical Chemistry

The classical view of entropy plays a role in chemical reactions, and is summarized as:

ΔS = (ΔH -Δ F)/T      (4)

where:

H = enthalpy
F = free energy or Gibb’s free energy
T = temperature

Note that this is very similar to equation (3). Chemists are less interested in the state of a "static" system, as equation (1) describes for the physicist. Chemists want to know whether a given chemical reaction will take place. In equation 4, S is interpreted as an incremental change in the entropy of the chemical system, in the event of a chemical reaction.
Rearranging equation (4) gives:

ΔF = (ΔH - TΔS)        (5)

The value of F which tells us whether a given chemical reaction will go forward spontaneously, or whether it needs to be pumped. The enthalpy, H, is the heat content of the system, and so the change in enthalpy, H, is the change in heat content of the system. If that value is smaller than TS, then F will be negative, and the reaction will proceed spontaneously; the TS term represents the ability to do the work required to make the reaction happen. However, if F is positive, such that H is greater than TS, then the reaction will not happen spontaneously; we still need at least F worth of energy to make it happen.

Microscopic viewpoint (statistical mechanics)

In the second half of he nineteenth century, James Clerk Maxwell, Ludwig Boltzmann and Josiah Willard Gibbs extended the ideas of classical thermodynamics into a field now called statistical mechanics. Whereas classical thermodynamics dealt with single extensive systems, mechanics attempts to relate the macroscopic properties of an assembly of particles to the microscopic properties of the particles themselves. The temperature, for instance, of a system defines a macrostate, whereas the kinetic energy of each molecule in the system defines a microstate. The macrostate variable, temperature, is as an expression of the average of the microstate variables, an average kinetic energy for the system. Hence, if the molecules of a gas move faster, they have more kinetic energy, and the temperature naturally goes up. Statistical mechanics, as its name implies, is not concerned with the actual motions or interactions of individual particles, but investigates instead their most probable behavior.

Statistical mechanics explains entropy as the amount of uncertainty (or "mixedupness" in the words of Gibbs) which remains about a system, after its observable macroscopic properties have been taken into account. For a given set of macroscopic quantities, like temperature and volume, the entropy measures the degree to which the probability of the system is spread out over different possible quantum states. The more states available to the system with higher probability, and thus the greater the entropy.

Boltzmann first defined the general form of the definition of entropy in statistical mechanics:

S = -k•Σ[P_ilog(P_i)]     (6)

where:
P_i = the probability that particle "i" will be in a given microstate
k = Boltzmann's constant (1.380658×10-23 Joules/Kelvin)

Note that in contrast to equation (1), neither the temperature nor the heat energy appear explicitly in equation (6). But the restriction that all of the microstate probabilities must be calculated for the same macrostate, assures that, as in the earlier case, the system must be in a state of thermal equilibrium.

If all of the probabilities P_i are the same, equation (6) reduces to:

S = k•log(N)              (7)

where N which is the total number of microstates available to the system. N is not the total number of particles, but rather is the total number of microstates that the particles could occupy, with the constraint that all such microstate collections would show the same macrostate.

Entropy and Information Theory

It would be difficult to understate the importance of the concept of entropy beyond the physical sciences and engineering. One important application of entropy came in the field of information theory, a branch of the mathematical theory of probability and mathematical statistics that quantifies the concept of information, and explains the limits and efficiency of information processing. Claude Shannon, widely recognized as the founder of information theory showed that:

S = -k•Σ[Pilog(P_i)]

Note that this identical to equation (6) above. Shannon showed that the Boltzmann entropy is the only function which satisfies the requirements for a function to measure the uncertainty in a message (where a "message" is a string of binary bits). In this case, the constant k is recognized as only setting the units; it is arbitrary, and can be set equal to exactly 1 without any loss of generality (In this case the probability P_i is the probability for the value of a given bit (usually a binary bit, but not necessarily).

In Shannon information theory, the entropy is a measure of the uncertainty over the true content of a message, but the task is complicated by the fact that successive bits in a string are not random, and therefore not mutually independent, in a real message. Also note that "information" is not a subjective quantity here, but rather an objective quantity, measured in bits.

Entropy and Ecological Economics

Entropy would also play an important role in the field of ecological economics. Nicholas Georgescu-Roegen, a Romanian mathematician, statistician and economist, published The Entropy Law and the Economic Process (1971), which placed economic production in its biophysical context. Among pioneers of ecological economics, Georgescu went the furthest in exposing the shortcomings of specific conventional economic theories and specific economic tools such as production functions. His fame and notoriety on the subject stem from the sweeping claims he made about the constraints imposed by the entropy law, and the degree to which he is cited by many influential scholars in the field, such as Herman Daly, Robert Ayres, and Robert Costanza.

The testament to his fundamental insight is the degree to which thermodynamics—and more generally the analysis of energy and material flows—forms a cornerstone of the “pre-analytic vision” of ecological economics, as well as the empirical work of many of its practitioners. Georgescu’s claim that the entropy law formed the “taproot” of economic scarcity stemmed from a simple series of observations. The economic process is a work process and as such it is sustained by a flow of low entropy energy and matter from the environment. As materials and energy are transformed in production and consumption processes higher entropy waste heat and matter ultimately are released to the environment. The circular flow of exchange value, which grabs the spotlight in conventional economic analysis, is an intermediate step in the process powered by the unidirectional flow of energy and materials.

Sources

• Baaquie, Belal E., Laws of Physics : A Primer, National University of Singapore, Accessed 14 March 2010.
• Cleveland, Cutler J. and Matthias Ruth. When, Where, and By How Much Do Biophysical Limits Constrain the Economic Process? The Contribution of Nicholas Georgescu-Roegen to Ecological Economics. Ecological Economics, 22: 203-223 (1997).
• Fitzpatrick, Richard, Thermodynamics & Statistical Mechanics: An intermediate level course, The University of Texas at Austin, Accessed 14 March 2010.
• Greitzer, E. M. , Z. S. Spakovszky, I. A. Waitz, Thermodynamics and Propulsion, Massachusetts Institute of Technology, Accessed 14 March 2010.
• Thomson, Tim, The Definitions of Entropy, Accessed 14 March 2010.
• Wikipeida Contributors, Entropy, Wikipedia, Accessed 14 March 2010.