Jump to content

Entropy of Mixing


Recommended Posts

My scribblings came up with the following interesting function

f(x1,x2,... xn) = -ln(x1^x1*x2^x2*,... xn^xn)

Where:

x1+x2+... xn = 1

0<xi<1

I suspect that this function ranges 0 to ln(n) but a proof is beyond me.

Assistance would be most appreciated.

 

Link to comment
Share on other sites

4 hours ago, sethoflagos said:

My scribblings came up with the following interesting function

f(x1,x2,... xn) = -ln(x1^x1*x2^x2*,... xn^xn)

Where:

x1+x2+... xn = 1

0<xi<1

I suspect that this function ranges 0 to ln(n) but a proof is beyond me.

Assistance would be most appreciated.

 

It gets a min when one xi=1 and all others are 0. It gets a max when all xi are equal.

Link to comment
Share on other sites

13 minutes ago, Genady said:

It gets a min when one xi=1 and all others are 0. It gets a max when all xi are equal.

Yes, thank you, Genady. Trial and error seems to indicate so.

But how does one derive those values mathematically?

Link to comment
Share on other sites

9 minutes ago, sethoflagos said:

Yes, thank you, Genady. Trial and error seems to indicate so.

But how does one derive those values mathematically?

I'd start with simplifying the expression and then finding an extremum using Lagrange multiplier method.

Link to comment
Share on other sites

The approach above should give you the max(f())=ln(n) quite fast. To get min(f), assume that ln(xj) is the smallest of all ln(xi), take the condition

x1+x2+... xn = 1,

multiply it by -ln(xj):

-(x1+x2+... xn)*ln(xj)=-ln(xj).

Because all ln(xi)>=ln(xj),

-(x1*ln(x1)+...+xn*ln(xn))>=-ln(xj).

The left side is -ln(x1^x1*x2^x2*... xn^xn), the right side -ln(xj)>=0. Thus

-ln(x1^x1*x2^x2*... xn^xn)>=0.

And we know how to get it =0. Thus 0 is the minimum.

Link to comment
Share on other sites

  • 2 months later...
On 4/3/2022 at 11:53 PM, Genady said:

To get min(f), assume that ln(xj) is the smallest of all ln(xi), take the condition

x1+x2+... xn = 1,

multiply it by -ln(xj):

-(x1+x2+... xn)*ln(xj)=-ln(xj).

Because all ln(xi)>=ln(xj),

-(x1*ln(x1)+...+xn*ln(xn))>=-ln(xj).

The left side is -ln(x1^x1*x2^x2*... xn^xn), the right side -ln(xj)>=0. Thus

-ln(x1^x1*x2^x2*... xn^xn)>=0.

And we know how to get it =0. Thus 0 is the minimum.

I'm so sorry, Genady, I missed this last post of yours.

So for the maximum we get the Legrangian function

L(x1, x2 ... xn, k) = -x1.ln(x1) -x2.ln(x2) ... -xn.ln(xn) - k.(x1 +x2 ... +xn -1)

The partial derivatives for all xi = -(ln(xi) +k +1) and therefore are zero at -ln(xi) = k+1  or xi = e^(-k-1)

The partial derivative wrt k is zero for the constraint condition x1 +x2 ... +xn = 1

Hence n.e^(-k-1) = 1 which solves to k = ln(n) -1

Hence xi = 1/n and ln(xi) = -ln(n)

Hence max value of original function = n.(1/n.ln(n)) = ln(n)

On 4/3/2022 at 11:53 PM, Genady said:

The approach above should give you the max(f())=ln(n) quite fast.

And you were not wrong! Thankyou so much for putting a new string on my mathematical bow.

I think I'll go away now and quietly try this method out on a few other functions.

 

   

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.