Data Manipulation

Please make sure you have downloaded the workshop dataset. See Setup for instructions.

Our data

We will use data about global warming. Our overall research hypothesis (merely for didactics) are:

  • H01 the earth average temperature has not dramatically increased since the advent of electronics

  • H02 the emission of carbon dioxide does not influence the earth average temperature

Importing data from text

We will assign the data from GlobalLandTemperaturesByCountry.csv to a variable called mydata

Import dataset > From text (base)

Preview data and give it a nice name

You can also load your data with the read_csv command

Input

mydata <- read.csv("GlobalLandTemperaturesByCountry.csv")

You can inspect the first few lines of mydata with head

Input

head(mydata, 5)

Basic data commands

names( ): check variable names

Input

names(mydata)

head( ): View first n lines of the table

Input

head(mydata, n = 10)

table( ): check variable values and frequency

Input

table(mydata$Country)

Basic data management commands

is.factor( ): check if the variable is defined as categorical

Input

is.factor(mydata$Country)

as.factor( ): changes variable to categorical format

Input

mydata$Country <- as.factor(mydata$Country)

Input

is.factor(mydata$Country)

numeric( ): check if the variable is defined as numerical

Input

is.numeric(mydata$AverageTemperature)

Data management with “dplyr” package

Removing empty cells/rows

You might have noticed NA in some rows of the dataset, what are they?

Input

head(mydata, n = 10)

We can remove NAs with na.omit

Input

mydata <- na.omit(mydata)

Input

head(mydata, n = 10)

select ( ): selects columns based on columns names

Useful when your data has many columns and you only need a subset of them

Input

select(mydata, dt, Country)

filter ( ): selects cases based on conditions

Input

filter(mydata, Country=="Canada")

filter may accept more than one condition

Pipe | filter any row that matches either condition

Input

filter(mydata, Country=="Canada" | Country == "China")

Ampersand & filter any row that matches both conditions

Input

filter(mydata, Country=="Canada" & AverageTemperature > 12)

Adding new columns with mutate( )

We might need new columns representing operations on previously existing data. This is required if, for instance, we want to create a numeric variable named year or to create a categorical variable named era, which represents if the measurement was from the electronic or gas & oil era.

Adding column representing the year

Input

mydata <- mutate(mydata, year = as.numeric(format(as.Date(dt), "%Y")))

There are multiple commands packed in the mutate operation. Take a look at each transformation, step by step:

Input

format(as.Date(mydata$dt), "%Y")
as.numeric(format(as.Date(mydata$dt), "%Y"))
mutate(mydata, year = as.numeric(format(as.Date(dt), "%Y")))
mydata <- mutate(mydata, year = as.numeric(format(as.Date(dt), "%Y")))

Input

head(mydata)

is.numeric(mydata$year)

Adding column representing the industrial era

We use a if_else comparison. If year is less or equal to 1969, we assign the value gas & oil. Otherwise, we assign the value electronic.

Input

mydata <- mutate(mydata, era=if_else(
    year <= 1969, "gas & oil", "electronic",
))

head(mydata, n=5)

View in GitHub