Category: programming (Page 3 of 6)

Reading a folder with many small files

2019-04-14 / Luis

One of the tools we use in our research is NIR (Near-Infrared Spectroscopy), which we apply to thousands of samples to predict their chemical composition. Each NIR spectrum is contained in a CSV text file with two numerical columns: wavelength and reflectance. All files have the same number of rows (1296 in our case), which corresponds to the number of wavelengths assessed by the spectrometer. One last thing: the sample ID is encoded in the file name.

As an example, file A1-4-999-H-L.0000.csv’s contents look like:

8994.82461,0.26393
\n8990.96748,0.26391
\n8987.11035,0.26388
\n8983.25322,0.26402
\n8979.39609,0.26417
...

Gratuitous picture: tree on stilts (Photo: Luis, click to enlarge).

Calculating parliament seats allocation and quotients

2017-10-03 / Luis

I was having a conversation about dropping the minimum threshold (currently 5% of the vote) for political parties to get representation in Parliament. The obvious question is how would seat allocation change, which of course involved a calculation. There is a calculator in the Electoral Commission website, but trying to understand how things work (and therefore coding) is my thing, and the Electoral Commission has a handy explanation of the Sainte-Laguë allocation formula used in New Zealand. So I had to write my own seat allocation function: Continue reading

New Zealand Electoral Commission results website. It held really well in election night.

Collecting results of the New Zealand General Elections

2017-10-02 / Luis

I was reading an article about the results of our latest elections where I was having a look at the spatial pattern for votes in my city.

I was wondering how would I go over obtaining the data for something like that and went to the Electoral Commission, which has this neat page with links to CSV files with results at the voting place level. The CSV files have results for each of the candidates in the first few rows (which I didn’t care about) and at the party level later in the file.

Mobile phone antenna in church (Photo: Luis, click to enlarge).

Functions with multiple results in tidyverse

2017-09-07 / Luis

I have continued playing with the tidyverse for different parts of a couple of projects.

Often I need to apply a function by groups of observations; sometimes, that function returns more than a single number. It could be something like for each group fit a distribution and return the distribution parameters. Or, simpler for the purposes of this exploration, calculate and return a bunch of numbers.
Continue reading

Turtles all the way down

2017-09-01 / Luis

One of the main uses for R is for exploration and learning. Let’s say that I wanted to learn simple linear regression (the bread and butter of statistics) and see how the formulas work. I could simulate a simple example and fit the regression with R:
Continue reading