Jump to content
Sign in to follow this  
Prometheus

Random Vectors and their inner product.

Recommended Posts

Let's say i have two random vectors, X and Y, both of length n*. The elements of either vector are not independent, and X and Y themselves are not independent. All elements are non-negative.

I want to explore the behaviour of the inner product of X and Y as 9e4876601ed31f29df7cf71a15538506-1.gif.

Not sure where to begin though: does anyone know of any good references/advice to get me started?

 

I'm sure i once saw a paper that showed if the RVs are independent then the inner product converges to zero, but can't for life of me dig it up: that'd be a good start.

 

Cheers.

 

*Edit: i meant of n dimensions.

Edited by Prometheus

Share this post


Link to post
Share on other sites

If all elements have to be positive or zero, the inner product can only be zero if at every index one of the vectors is zero, which is not very random.

Share this post


Link to post
Share on other sites

Let's say i have two random vectors,

In how many dimensions?

 

X and Y, both of length n.

So,

if they're 3D, Xx,Xy,Xz and Yx, Yy, Yz,

you know from the start that:

 

sqrt(Xx^2+Xy^2+Xz^2)=sqrt(Yx^2+Yy^2+Yz^2)

sqrt(Xx^2+Xy^2+Xz^2)=n

sqrt(Yx^2+Yy^2+Yz^2)=n

 

therefor

 

Xx^2+Xy^2+Xz^2=Yx^2+Yy^2+Yz^2

 

In 2D:

 

sqrt(Xx^2+Xy^2)=sqrt(Yx^2+Yy^2)

sqrt(Xx^2+Xy^2)=n

sqrt(Yx^2+Yy^2)=n

 

Xx^2+Xy^2=Yx^2+Yy^2

 

so once we randomize the first Xx:

Xx = sqrt(Yx^2+Yz^2-Xy^2) // Xx = rand();

 

then we cannot randomize Xy, because it has to have length equal n

Xx = (float) rand() / MAX_INT;

sqrt(Xx^2+Xy^2)=n

Xx^2+Xy^2=n^2

Xy^2=n^2-Xx^2

Xy=sqrt(n^2-Xx^2) // either n and Xx are known already.. then how can we randomize Xy.. ?

 

 

Normalization of vector is division each component by it's length

Xx / n = Xx'

Xy / n = Xy'

Yx / n = Yx'

Yy / n = Yy'

 

Length will be 1.

Why going n to infinity, if they will be just vector in 2D in range 0.0...1.0 per component, multiplied by n.. ?

 

Not sure where to begin though: does anyone know of any good references/advice to get me started?

I would start from writing small C/C++ program that would randomize vectors in loop,

and draw them on screen, to be able to analyze them.

Edited by Sensei

Share this post


Link to post
Share on other sites

 

Edit: I think if the magnitude of the vectors is kept constant, the inner products would converge to zero.

 

By "length" do you mean dimension or magnitude?

Share this post


Link to post
Share on other sites

Very sorry to have wasted your time but i mis-stated the problem: n is the number of dimensions, not the length of the vectors. I'll edit that.

 

I did think about running a simulation, but thought i'd check if there were any analytic work on it first. Plus i work in matlab and that's slow.....

Share this post


Link to post
Share on other sites

The magnitude is not restricted at all.

 

Btw, the case where it converges to zero was not with vectors with non-negative elements.

Share this post


Link to post
Share on other sites

About Matlab: do you use a for-loop? Matlabs for-loop is notoriously slow. You have to combine the vectors and use matrix calculations instead. This is much faster.

 

I don't see how it could converge to zero. I think the series would look like a 1D random walk.

Share this post


Link to post
Share on other sites

The magnitude is not restricted at all.

 

So each component range is 0...+infinity, right?

 

Btw, the case where it converges to zero was not with vectors with non-negative elements.

 

 

I don't see how it could converge to zero. I think the series would look like a 1D random walk.

 

If vector component, in f.e. 2D is x,y, can be in range 0...+infinity

Then dot product will be

(+infinity) *(+infinity) + (+infinity)*(+infinity) + .... (repeat as many as dimensions)

 

I made such little C++ code:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, int *argv[] )
{
    srand( clock() );
    float max_value = 1000000;
    int max_i = 1000000;
    for( int i = 0; i < max_i; i++ )
    {
        float xx = (float) rand() * max_value / RAND_MAX;
        float xy = (float) rand() * max_value / RAND_MAX;
        float yx = (float) rand() * max_value / RAND_MAX;
        float yy = (float) rand() * max_value / RAND_MAX;

        float dot = xx * yx + xy * yy;
        printf( "%f\n", dot );
    }
    return( 0 );
}

After using it from command-line,

It generated series of rows (,exe >data,csv), that can be loaded from OpenOffice, and sorted, and show graph (max_value=1000):

 

post-100882-0-77263600-1492630038_thumb.png

 

3-dimensions version:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main( int argc, int *argv[] )
{
    srand( clock() );
    float max_value = 1000000;
    int max_i = 1000000;
    for( int i = 0; i < max_i; i++ )
    {
        float xx = (float) rand() * max_value / RAND_MAX;
        float xy = (float) rand() * max_value / RAND_MAX;
        float yx = (float) rand() * max_value / RAND_MAX;
        float yy = (float) rand() * max_value / RAND_MAX;
#if 1
        float zx = (float) rand() * max_value / RAND_MAX;
        float zy = (float) rand() * max_value / RAND_MAX;
#endif
        float dot = xx * yx + xy * yy;
#if 1
        dot += zx * zy;
#endif
        printf( "%f\n", dot );
    }
    return( 0 );
}

Graph from 3D version (max_value =1,000,000)

post-100882-0-15870400-1492631925_thumb.png

Edited by Sensei

Share this post


Link to post
Share on other sites

Hilbert Sequence Space (symbol l2) is an infinite dimensional vector space with a defined inner product.

 

Rn and Cn are not infinite dimensional since n is defined as a number and therefore cannot be infinite.

 

https://en.wikipedia.org/wiki/Hilbert_space

 

Edit

note you will need to scroll down to the section about Hilbert sequence space, the first part of the Wiki article is about finite dimensional Hilbert spaces.

Edited by studiot

Share this post


Link to post
Share on other sites

 

So each component range is 0...+infinity, right?

 

Correct.

 

 

I made such little C++ code:

 

Sorry, you lost me with this code: what are you up to?

Share this post


Link to post
Share on other sites

Rn and Cn are not infinite dimensional {vector spaces} since n is defined as a number and therefore cannot be infinite.

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

Share this post


Link to post
Share on other sites

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

Jeez man that's not right. What is the dimension of [math]\mathbb R^2[/math]?

 

If you're thinking of [math]\mathbb R^{\mathbb N}[/math], the direct product of countably many copies of the reals; or else perhaps [math]\oplus_{\mathbb N} \mathbb R[/math], the direct sum of countably many copies of the reals, both of those are infinite-dimensional vector spaces.

 

The direct sum is the set of functions on the naturals that are zero at all but finitely many places. The direct product doesn't have that restriction. It's like the distinction between the collection of formal polynomials versus the collection of formal power series.

 

The direct sum has countably infinite dimension. The direct product can not possibly have countably infinite dimension. Why is that? Because linear combinations are defined as being finite. So the obvious basis [math]\{e_i\}[/math] doesn't work for the direct product, while it does for the direct sum.

Edited by wtf

Share this post


Link to post
Share on other sites

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

 

Being a simple soul, I like to look at it this way.

 

 

n is the dimension of the vector space.

 

n is number; no number is infinite.

 

hence n is finite, however large.

 

This is the old chestnut confusion of the difference between the cardinality of a set and the maximum value (or otherwise) of an element.

 

Nice artwork by the way, it won't display those symbols here for me.

Share this post


Link to post
Share on other sites

Nice artwork by the way, it won't display those symbols here for me.

Math markup works fine for me on this site.

[math]\mathbb C[/math]

renders as [math]\mathbb C[/math]. Doesn't work for you?

Share this post


Link to post
Share on other sites

I write formulae in MathType and copy/paste.

 

So yes the symbols are available in MathType, but do not paste into this forum.

Matrices have to be adjusted as well, but a lot of other stuff works just fine.

Share this post


Link to post
Share on other sites

Nice talkin' to y'all over the years on this site. I'm probably going to get banned by humorless scolds who can't read through a tongue-in-cheek post to the deeper more serious point, which I then explained in detail.

 

All the best.

Share this post


Link to post
Share on other sites

No, I don't think this is quite correct. If [math]n \in \mathbb{N}[/math] and [math]\mathbb{N}[/math] is infinite but countable i.e has cardinality [math]\aleph_0[/math], then the vector spaces [math]\mathbb{R}^n[/math] and [math]\mathbb{C}^n[/math] are necessarily infinite dimensional vector spaces.

Good God, did I really say this?

 

It appears that I did - I cannot imagine what I was thinking, as it is quite clearly nuts.

 

Apologies

Edited by Xerxes

Share this post


Link to post
Share on other sites

No problems we are all friends here and friends help when someone has had too much tequila.

 

:)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.