Linux code for testing astrology claims about "transits"

[MonDie]

I have my first question, so I might as well start the thread now.

 

I began this project long, long ago, and I might as well complete it even though my interest has waned. I stopped looking at the planets' current longitudes to avoid bias, and I began writing down every day which I thought might be a Venus "transit" or Mars "transit". Two more analogous sets of dates were acquired from a journal I had kept for a few years. I won't share the dates publicly, but I'll share the results. In this instance, different outcomes for the different data sets could shed light on my bias.

I also want to share the code, so testing these claims becomes easier for everyone. With such means freely available during my youth, I might have relinquished my belief sooner and with more confidence, and spent more time studying science!

 

 

First question. SOLVED

 

read foo

until [[ foo == "end" ]];

do

line=$(cat ~/Desktop/merc | grep $foo)

line=$(echo $line | sed "s/-[0-9]\./--/")

echo $line

longitude=$(echo $line | tr " " "\n" | grep [0-9][.])

echo $longitude

read foo

done

This is Bash code, what you type into the Linux shell. ~/Desktop/merc is the path to my plain text document downloaded from NASA HORIZONS. I only had #31 selected for table options.

http://ssd.jpl.nasa.gov/horizons.cgi#top

I need to assign the variable only the longitude. Unfortunately the longitude and latitude are sometimes indistuighablethe above code only works when latitude is negative. I might need a utility that prints a line based on its ordinal position. I am learning c++, so I might eventually include some of those libraries. Can anyone help?

 

Okay solved it. The sed utility, formatted "s/input/output/", only changes the first instance of the input text. Thus I set it to change the first decimal-point to a comma, then grepped the comma.

 

line=$(echo $line | sed "s/\./\,/")

longitude=$(echo $line | tr " " "\n" | grep [,])

 

What comes next depends on how the math will be worked out, which I haven't attended to yet. I may have this script do the math, or I may have a separate script perform the math on its output. I only know some discrete probability, but math is where I excelled

Will start contemplating the math.

Edited July 24, 2015 by MonDie

[MonDie]

I just vastly simplified the code by combining all those greps, trs, and seds into one line. Additionally, now it makes a list, which could be used for the math after the loop is terminated with "end". One problem: "end" doesn't terminate the loop.

#! /bin/bash

read date
list=""
until [[ date == "end" ]];
do
## Greps the table, then isolates and assigns the longitude value(s).
longitude=$(cat ~/astr/merc | grep $date | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")
## Echos the longitude(s), then awaits user decision before appending $list.
echo $longitude
read decision
if [[ $decision != "undo" ]]
then list=$list" "$longitude
fi
## Echoes $list for you to see.
echo $list

## Obtain next date or end loop signal.
read date
done

## Indicate completion of the until loop.

echo "List completed."

Another problem is that $list is a string. During the math, the content must be read not as a string, but as floating point value(s). http://stackoverflow.com/questions/6348902/how-can-i-add-numbers-in-a-bash-script Before this can happen, I'll need to isolate the desired number from $list, which may require me to number the longitudes or else I'll need some sort of grep-like utility that relies on ordinal position.
Here's an alternative version that numbers each successive entry.

#! /bin/bash

read date
count=1
list=""
until [[ date == "end" ]];
do
## Greps the table, then isolates and assigns the longitude value(s).
longitude=$(cat ~/astr/merc | grep $date | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")
## Echos the longitude(s), then awaits user decision before appending $list.
echo $longitude
read decision
if [[ $decision != "undo" ]]
then longitude=$count"-"$longitude
list=$list" "$longitude
fi
## +1 count
count=$(($count + 1))
## Echoes $list for you to see.
echo $list
## Obtain next date or end loop signal.
read date
done
## Indicate completion of the until loop.
echo "List completed."

Then the entries appear like this: 1-53.897867 2-183.3786478 3-321.6738923
One can grep the desired entry with the code below:

floatingpoint=$(echo $list | tr " " "\n" | grep [$number][-] )
## $number is the number of the desired entry. Putting the dash in brackets is similar to escaping it with the back-slash.

 

---

 

Regarding the mathematics:

At first I worried about getting the program to work in degrees, where 360 is followed by another 0. Fortunately, "aspects" and "orbs" are arbitrated differently from one astrologist to the next. Without excessive detail, an aspect is a specific longitudinal distance that, when achieved between two points or two celestial bodies, supposedly contributes to the effect. Examples include the conjunction (0°), trine (120°), and many more. The orb is the allowable error for a given aspect or celestial body. These are by no means fixed, so I think each user should arbitrate which degree ranges constitute a hit or a miss. This only allows for aspects between the celestial body's current position and the fixed "natal planets", but nearly all astrologists accept the validity of such, so it should be enough for a rudimentary test.

Again, it will greatly simplify matters since the program will not have to do degree calculations. Furthermore we'll only be dealing with the relatively simple binomial probability distribution. I might let the user weight some hits differently since the conjunction (0°) and opposition (180°) are often considered stronger, as are more precise aspect formations. Unless there is a better way, I propose doing two binomial probability calculations, one with more stringent success criteria, and averaging their results.

Edited July 24, 2015 by MonDie

[MonDie]

Okay, this is bizarre. It's going down the entire ephemeris and printing all of the longitudes, repeatedly, endlessly, so the only way it ceases is with Ctrl+C.

I decided to have the program read the dates off from a saved file, so they only need be entered once. The program is supposed to read each successive date by changing the first dash in the next date to a "z" by which grep can distinguish it, then removing that line with grep -v z. The z in $date gets changed back to a dash. In the dates file, the dates are separated by underscores which it translates to new lines.

 

 

#! /bin/bash

dlist=$(cat ~/astr/dates)
echo $dlist
list=""
until [[ dlist == "" ]];
do
## Assigns the next value in $dlist to $date, then removes it from $dlist.
date=$(echo $dlist | sed "s/-/z/" | tr "_" "\n" | grep z | sed "s/z/-/")
echo $date
dlist=$(echo $dlist | sed "s/-/z/" | tr "_" "\n" | grep -v z)
echo $dlist
## Greps the table, then isolates and assigns the longitude value(s). $date is quoted so grep can handle whitespace.
longitude=$(cat ~/astr/merc | grep "$date" | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")
echo $longitude
## Appends $longitude to $list.
list=$list" "$longitude
echo $list
done
## Indicate completion of the until loop.
echo "List completed."

 

I already copied the code to another laptop, and it had the same runtime error. If you want to run it yourself, however, it's real easy to do.

Just put it in a text file, make it executable

chmod +0111 /path/to/file

then run it via the terminal

/path/to/file

Edited July 28, 2015 by MonDie

[Endy0816]

Pure guess work, but shouldn't there be another dollar sign on line 13?

 

Edit: Looks like you caught it.

 

So the loop isn't completing now...

 

Something has to be going on with dlist.

Edited July 28, 2015 by Endy0816

[MonDie]

Pure guess work, but shouldn't there be another dollar sign on line 13?

 

Edit: Looks like you caught it.

 

So the loop isn't completing now...

 

Something has to be going on with Dlist.

 

It's more than just an infinite loop. It's going through every single line of the ephemeris obtained from the NASA site (~/astr/merc), but it should only go through those lines that match the dates in ~/astr/dates.

 

I made an altered script for debugging. It prints the variable name before each variable so we can identify them, and it has a read random before done to halt the script before each loop. Apparently it assigns nothing to longitude on the first round, and nothing to longitude or dlist on the second round. On the third round, dlist is empty—which doesn't terminate the until loop for some reason—, so nothing gets assigned to date. Because $date is empty, grep selects the entire ephemeris, and the program assigns all the longitudes to $longitude at once, and the program repeatedly appends this to $list.

 

#! /bin/bash

 

dlist=$(cat ~/astr/dates)

echo "initial dlist "$dlist

list=""

until [[ dlist == "" ]];

do

## Assigns the next value in $dlist to $date, then removes it from $dlist.

date=$(echo $dlist | sed "s/-/z/" | tr "_" "\n" | grep z | sed "s/z/-/")

echo "date "$date

dlist=$(echo $dlist | sed "s/-/z/" | tr "_" "\n" | grep -v z)

echo "dlist "$dlist

## Greps the table with $date, then isolates and assigns the longitude value(s).

longitude=$(cat ~/astr/merc | grep "$date" | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")

echo "longitude "$longitude

## Appends $longitude to $list.

list=$list" "$longitude

read random

done

## Indicate completion of the until loop.

echo "List completed."

---

 

I've spotted it! The problem with longitude was a faulty dates file. I entered Jan-1 instead of Jan-01, which appears nowhere in the ephemeris.

 

The problem with dlist is that I'm using underscores as new lines. New lines become whitespace when they're stored in a variable. This is okay when those new lines began as whitespace anyway, but not when those new lines began as underscores. I'll see how I can fix this.

 

edit: Furthermore, the until loop wasn't completing because I forgot the $ in [[ $dlist == "" ]];

 

---

 

I fixed it. The dates file now separates entries with whitespace or new line, and within entries you must use a greater-than (>) where whitespace would be.

 

#! /bin/bash

 

dlist=$(cat ~/astr/dates)

echo "initial dlist "$dlist

list=""

until [[ $dlist == "" ]];

do

## Assigns the next value in $dlist to $date, then removes it from $dlist.

date=$(echo $dlist | sed "s/-/z/" | tr " " "\n" | grep z)

date=$(echo $date | sed "s/z/-/" | tr ">" " ")

echo "date "$date

dlist=$(echo $dlist | sed "s/-/z/" | tr " " "\n" | grep -v z)

echo "dlist "$dlist

## Greps the table with $date, then isolates and assigns the longitude value(s).

longitude=$(cat ~/astr/merc | grep "$date" | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")

## Appends $longitude to $list.

list=$list" "$longitude

echo $list

done

## Indicate completion of the until loop.

echo "List completed."

Time for some math! Wooh! Not sarcasm!

Again, I want some feedback on the idea of doing two binomial probability distibutions, one with more stringent success criteria, and then average their results. This is to give more weight to more exact "aspect" formations.

The sad part is that this probably could be done more easily with arrays. Unfortunately I've never tried arrays in bash, and I never got them to work in C++. heh

Edited July 28, 2015 by MonDie

[Endy0816]

 

Time for some math! Wooh! Not sarcasm!

 

Always a joyous occasion when you finally get a program working

 

 

The sad part is that this probably could be done more easily with arrays. Unfortunately I've never tried arrays in bash, and I never got them to work in C++. heh

 

Oddly enough I was thinking along similar lines while mulling this problem over in my head.

 

 

I'm still not sure exactly what claim you are trying to disprove. Keeping in mind my total knowledge of astrology can be summed up as, "prognostication based on birth date and planetary position".

[MonDie]

I'm still not sure exactly what claim you are trying to disprove. Keeping in mind my total knowledge of astrology can be summed up as, "prognostication based on birth date and planetary position".

I'm testing my past self's claim that I could, without peeking, identify the days when Venus or Mars was contacting my natal chart. I am however creating a more flexible and 100% free program that can test an array of similar claims. The user generates the ephemeris from nasa.gov and thus can choose any celestial body, and they define what longitude ranges count as successes or failures in their binomial probability distributions.

[billiards]

Hi, just some (hopefully helpful) comments about your code, bear in mind I don't have access to your files (too lazy to download them myself) so the suggestions I have will probably need tweaking:

 

It looks like you are using some tricks that you figured out for yourself like the idea of replacing the '.' with a ',' grepping it out and then changing it back again. While these are very clever, they're unnecessarily complicated and would be considered bad "style" because they make the code hard to read and may cause unintended side effects. If longitude is always in the same column there are useful tools such as "awk" (or the vastly improved gawk) that will do a good job for you.

 

I also don't like the way you are looping using an until statement and modify your variable with some quite clunky code inside the loop. The whole thing can be done much more elegantly using a for statement:

#! /bin/bash

dlist=$(cat ~/astr/dates)
echo "initial dlist "$dlist

list=""
until [[ $dlist == "" ]];
do
## Assigns the next value in $dlist to $date, then removes it from $dlist.
date=$(echo $dlist | sed "s/-/z/" | tr " " "\n" | grep z)
date=$(echo $date | sed "s/z/-/" | tr ">" " ")

# The above could probably be made more concise (and readable) using something like:
# for date in $(cat ~/astr/dates) ; do

echo "date "$date
dlist=$(echo $dlist | sed "s/-/z/" | tr " " "\n" | grep -v z)
echo "dlist "$dlist
## Greps the table with $date, then isolates and assigns the longitude value(s).
longitude=$(cat ~/astr/merc | grep "$date" | sed "s/\./\,/" | tr " " "\n" | grep [,] | tr "," ".")

# There might be an easier way to do this, e.g. (where Y is the column number containing longitude):
# longitude=$(cat ~/astr/merc | grep "$date" | awk '{print $Y}')

## Appends $longitude to $list.
list=$list" "$longitude
echo $list
# Is this "list" of longitudes getting longer and longer and you're printing it out every iteration? Why? Lots of duplication!
done
## Indicate completion of the until loop.
echo "List completed."

It looks to me like the whole thing can be done far more quickly and concisely with a single call to awk.

#!/bin/bash

# Print the date and corresponding longitude for each date in the date file.

datefile=~/astr/dates
mercfile=~/astr/merc

gawk -v datefile=$datefile -v mercfile=$mercfile \
'FILENAME==datefile { dates[$1]=$1; next} \
 FILENAME==mercfile { if (($X) in dates) { \
                         print dates[$X],$Y}' \
$datefile $mercfile

# Where $X is the column number containing the datestring in the mercfile \
# and $Y is the column number containing the longitude in the mercfile.

If you want more help maybe you can post a head of the two files you are looking at to give some idea of what they look like.

 

Hope it helps.

Edited July 29, 2015 by billiards

[MonDie]

Most of the echoes are only there for debugging, and will be removed in the final. The code you provide is incomprehensible to me—I skipped learning awk because it looked complicated—, but I took some of your advice and made my own simplified script.

 

#! /bin/bash

list=""
for i in $(cat ~/astr/dates); do
## Assigns the next line of dates to $date via the i variable.
date=$(echo $i | tr "z" " ")

echo "date "$date
## Greps the table with $date, then isolates and assigns the longitude value(s).
longitude=$(cat ~/astr/merc | grep "$date" | awk '{print $3}')
## Appends $longitude to $list.
list=$list" "$longitude
echo $list
done
## Indicate completion of the until loop.
echo "List completed."

 

or, without the comments and excessive echoes,

 

#! /bin/bash

list=""
for i in $(cat ~/astr/dates); do
date=$(echo $i | tr "z" " ")
longitude=$(cat ~/astr/merc | grep "$date" | awk '{print $3}')
list=$list" "$longitude
done

 

Apparently the for loop runs line-by-line down the output of a command, in this case cat ~/astr/dates, assigning each successive line to $i. That was very convenient. I didn't even have to translate the whitespace back into new lines.

It was however interpretting whitespace as it does new lines, so you have to replace whitespace with a "z" in the dates file.

[billiards]

I'm glad to have helped

 

You could probably clean up that "z" trick. But hey -- if it works! -- it works and sometimes that's all that matters.

 

Learning the basics of awk is worthwhile (I notice you're using it in your cleaned up script to print $3). If you ever deal with very large text files learning how to use the arrays (as I demonstrated) will give your code some muscle.

[MonDie]

I found a trinomial distribution formula that's much the same to the binomial formula, but I'm not 100% sure that it's ordering the events by degree of success. I want to have strong successes and weak successes.

https://onlinecourses.science.psu.edu/stat414/node/106

Where the event types are A, B, and C, and lowercase denotes each's number of occurences, their formula goes like this.

 

[math]\frac{(a+b+c)!}{a! b! c!} P(A)^{a} P(B)^{b} P©^{c}[/math]

 

which is basically just an expanded version of the binomial formula which can be written as

 

[math]\frac{(a+b)!}{a! b!} P(A)^{a} P(B)^{b}[/math]

The question is, could I write it so that each A (strong success) also counts as a B (strong or weak success)?

Edited July 29, 2015 by MonDie

[billiards]

Let me see if I understand what you are trying to do correctly.

 

You want to see if your intuitive guesses about the positions of Mars and Venus match up to reality...

 

My recommendation would be to produce a plot. Plot a timeseries of the actual positions of the planets and then on top plot the points that you guessed. Then you will quickly be able to see if your points fall on the line or not. After that you can get a bit heavier on the statistics.

[MonDie]

Thanks, but can you provide some formulae used for such analyses?

I think I can handle degree differences after all. I will just have it change the math whenever the linear difference is over 180, so instead It'll add to the lower value the distance from the higher value to 360.

I could have it calculate, for each longitude, its distance from all points and record the least value (LV), after weighting some points more strongly by dividing the distance from them by a constant.

The program could generate a random dataset for comparison by simply grepping the ephemeris randomly. This would eliminate any potential effect of the time range. Suppose they choose a year where Mars happens to retrograde over their hot spot, giving the illusion of accurate guessing. Generating a random data set could prevent this.

To normalize our data, we would convert each LV to a p-value by counting how many of the randomized LVs fell above or below that value. Some of the randomized LVs will be equal to it: half should be distributed above and half below because, assuming the null hypothesis, this is what would be true on average if we could request more precise values.

 

After that it's very simply. I believe it goes like this, but I'm unsure whether we want the z-score or a t-test.

Where [math]\bar{x}[/math] and [math]n[/math] are the average LV p-value and the sample size, and [math]\sigma[/math] and [math]\mu[/math] are the standard deviation and mean of our random dataset:

 

[math]z=\frac{\bar{x}-\mu}{\sigma/\sqrt{n}}[/math]

 

Then we just convert the z-score to a p-value.

Edited July 30, 2015 by MonDie

[MonDie]

I'm trying to transition to C++ for the math, but I can't embed a bash script. I contemplated asking the user to copy and paste the entire longitudes list, but the list could get really long. I'm trying unsuccessfully to embed a small script that will execute the bash script from a separate file ( $HOME/astr/pr ).

I found this forum thread, and I'm attempting Yoda's advice. http://www.unix.com/programming/216190-putting-bash-script-c-program.html

I'm about to read this for more info. http://www.cplusplus.com/forum/beginner/13519/

 

#include<stdio.h>
#include<stdlib.h>

#include<iostream>
#include<fstream>
#include<sstream>
#include<string>
#include<cmath>

#define SHELLSCRIPT "\
#! /bin/bash \n\
$HOME/astr/pr \n\

echo \"Testing testing!\" \n\
"

int main() {
double x = 1.2;
std::cout<<x;
}

 

 

The last part is just to test the C++ include statements, and it works, but nothing in the bash script works.

Edited August 3, 2015 by MonDie