Read case data

Last updated on 2026-09-25 | Edit this page

Estimated time: 30 minutes

Overview

Questions

Where do you usually store outbreak data?
What data formats do you commonly use for analysis?
Can you import data directly from servers and health information systems?

Objectives

Identify common sources of outbreak data.
Import outbreak data from multiple formats into R environment.
Access and retrieve data from remote servers and health information systems using APIs.

Prerequisite

Prerequisites

This episode requires you to be familiar with Data science: Basic tasks with R.

Introduction

The first step in outbreak analysis is importing your dataset into the R environment. Data can come from local sources, like files on your computer, or external sources, like databases and health information systems (HIS).

Outbreak data takes many forms. It may be sorted as a flat file in various formats, housed in relational database management systems (RDBMS), or managed through specialized HIS like SORMAS and DHIS2. These HISs offer application programming interfaces (APIs) that allow authorized users to modify and retrieve data entries efficiently, making them particularly valuable for large-scale institutional health data collection and storage.

This episode demonstrates how to read case data from each of these sources. Let’s begin by loading the packages we’ll need. We will use rio to read data stored in files and readepi to access data from RDBMS and HIS. We will also load here to locate file paths within your project directory, and tidyverse, which includes magrittr (providing the pipe operator %>%) and dplyr (for data manipulation). The pipe operator allows us to chain functions together seamlessly.

R

# Load packages
library(tidyverse) # for {dplyr} functions and the pipe %>%
library(rio) # for importing data from files
library(here) # for easy file referencing
library(readepi) # for importing data directly from RDBMS or HIS
library(dbplyr) # for a database backend for {dplyr}

Checklist

The double-colon (`::`) operator

The double-colon :: in R lets you call a specific function from a package without loading the entire package. For example, dplyr::filter(data, condition) uses the filter() function from the dplyr package, without requiring library(dplyr).

This notation serves two purposes: it makes code more readable by explicitly showing which package each function comes from, and it prevents namespace conflicts that occur when multiple packages contain functions with the same name.

Prerequisite

Setup a project and folder

Create an RStudio project. If needed, follow this how-to guide on “Hello RStudio Projects” to create one.
Inside the RStudio project, create a data/ folder.
Download ebola_cases_2.csv and marburg.zip CSV files, and save them inside the data/ folder.

Reading from files

Several packages are available for importing outbreak data stored in individual files into R. These include {rio}, {readr} from the tidyverse, {io}, {ImportExport}, and {data.table}. Together, these packages offer methods to read single or multiple files in a wide range of formats.

The below example shows how to import a csv file into R environment using the rio package. We use the here package to tell R to look for the file in the data/ folder of your project, and dplyr::as_tibble() to convert into a tidier format for subsequent analysis in R.

R

# read data
# e.g., if the path to our file is "data/raw-data/ebola_cases_2.csv" then:
ebola_confirmed <- rio::import(
  here::here("data", "raw-data", "ebola_cases_2.csv")
) %>%
  dplyr::as_tibble() # for a simple data frame output

# preview data
ebola_confirmed

OUTPUT

# A tibble: 120 × 4
    year month   day confirm
   <int> <int> <int>   <int>
 1  2014     5    18       1
 2  2014     5    20       2
 3  2014     5    21       4
 4  2014     5    22       6
 5  2014     5    23       1
 6  2014     5    24       2
 7  2014     5    26      10
 8  2014     5    27       8
 9  2014     5    28       2
10  2014     5    29      12
# ℹ 110 more rows

You can use the same approach to import other file formats such as tsv, xlsx, and more.

Checklist

Why should we use the {here} package?

The here package is designed to simplify file referencing in R projects by providing a reliable way to construct file paths relative to the project root. The main reason to use is for cross-environment compatibility.

It works across different operating systems (Windows, Mac, Linux) without needing to adjust file paths.

On Windows, paths are written using backslashes ( \ ) as the separator between folder names: "data\raw-data\file.csv" .
On Unix based operating systems such as macOS or Linux the forward slash ( / ) is used as the path separator: "data/raw-data/file.csv".

The here package reinforces the reproducibility of your work across multiple operating systems. If you are interested in reproducibility, we invite you to read this tutorial to increase the openess, sustainability, and reproducibility of your epidemic analysis with R

Challenge

Reading compressed data

Can you read data from a compressed file in R?

Download this zip file containing Marburg outbreak data and then import it to your R environment.

Give me a hint

You can check the full list of supported file formats in the rio package on the package website. To see the list of supported formats in rio, run:

R

rio::install_formats()

Show me the solution

R

rio::import(here::here("data", "Marburg.zip"))

Reading from databases

The readepi library contains functions that allow you to import data directly from RDBMS. The readepi::read_rdbms() function supports importing data from servers such as Microsoft SQL, MySQL, PostgreSQL, and SQLite. It build on the {DBI} package, which provides a general interface for interacting RDBMS.

Discussion

Advantages of reading data directly from a database?

Importing data directly from a database optimizes the memory usage in the R session. By processing the database with “queries” (e.g., SELECT, FILTER, GROUP BY) before extraction, you reduce the memory load in our RStudio session. In contrast, loading an entire dataset into R for manipulation can consume more RAM than your local machine can handle, potentially causing RStudio to slow down or freeze.

RDBMS also enable multiple users to access, store, and analyze parts of the dataset simultaneously without transferring individual files. This eliminates the version control problems that arise when multiple file copies circulate among users.

1. Connect with a database

You can use the readepi::login() function to establish a connection to the database, as shown below:

R

# establish the connection to a test MySQL database
rdbms_login <- readepi::login(
  from = "genome-mysql.soe.ucsc.edu",
  type = "MySQL",
  user_name = "genome",
  password = "",
  driver_name = "",
  db_name = "hgFixed",
  port = 3306
)

OUTPUT

✔ Logged in successfully!

R

rdbms_login

OUTPUT

<Pool> of MySQLConnection objects
  Objects checked out: 0
  Available in pool: 1
  Max size: Inf
  Valid: TRUE

The function parameters are:

from: The database server address (genome-mysql.soe.ucsc.edu)
type: The type of database system (“MySQL”)
user_name: The username for authentication (“genome”)
password: The password (empty string “” indicates no password required for this public test database)
driver_name: The database driver (empty string uses the default driver)
db_name: The specific database to connect to (“hgFixed”)
port: The port number for the connection (3306)

The function returns a connection object stored in variable rdbms_login, which can then be used to query and retrieve data from the database.

Callout

Note: This example uses a public test database from the University of California, Santa Cruz (UCSC) Genome Browser project, which is why no password is required. It uses the standard MySQL port (3306), so it is less likely to be blocked than test servers on non-standard ports, but access could still be limited by strict organizational network restrictions.

2. Access the list of tables from the database

The readepi::show_tables() function retrieves the full list of table names from a database:

R

# get the table names
tables <- readepi::show_tables(login = rdbms_login)

tables[1:5]

OUTPUT

[1] "affy10KDetails"  "affy120KDetails" "affyExps"        "affyGenoDetails"
[5] "arbFlyLifeAll"

In a relational database, you typically have multiple tables. Each table represents a specific entity (e.g., patients, care units, treatments). Tables are linked through common identifiers called primary keys or foreign keys.

3. Read data from a table in a database

You can read the data from the author table using dplyr::tbl(). This table lists authors of GenBank sequence submissions, with each name stored as "LastName,F.I.".

R

# import data from the 'author' table using an SQL query
dat <- rdbms_login %>%
  dplyr::tbl(from = "author") %>%
  dplyr::filter(stringr::str_sub(string = name, start = 1, end = 1) == "A") %>%
  dplyr::slice_sample(n = 20) %>% 
  dplyr::arrange(desc(id))

dat

OUTPUT

# A query:    ?? x 3
# Database:   mysql 5.5.5-10.11.8-MariaDB [@genome-mysql.soe.ucsc.edu:/hgFixed]
# Ordered by: desc(id)
       id name                                                               crc
    <int> <chr>                                                            <dbl>
 1 360722 Aerbajinai W, Ghosh MC, Liu J, Kumkhaek C, Zhu J, Chin K, Roua… 9.96e8
 2 329162 Antony J, Zanini E, Kelly Z, Tan TZ, Karali E, Alomary M, Jung… 2.21e9
 3 318804 Abdullah NH and Ismail S.                                       6.18e7
 4 314722 Abeyta A, Castella M, Jacquemont C and Taniguchi T.             3.98e9
 5 303662 Anderson ST, Barclay JL, Fanning KJ, Kusters DH, Waters MJ and… 3.93e9
 6 258335 Alain,K., Rolland,S., Crassous,P., Lesongeur,F., Zbinden,M., l… 3.99e9
 7 255427 Aklujkar,M., Young,N.D., Holmes,D., Chavan,M., Risso,C., Kiss,… 9.91e8
 8 253899 Ammanamanchi,N., Katz,J. and Mead,J.                            2.79e9
 9 243756 Ajith,H., Vyas,D.H. and Patell,V.M.                             9.92e8
10 211833 Aquino,J.D., Tang,W.F., Ishii,R., Ono,T., Eshita,Y., Aono,H. a… 3.82e9
11 207322 Alexeev,D., Chukin,M.M., Generozov,E.V., Govorun,V.M., Gribano… 1.69e9
12 173806 Ahlquist,P., Dasgupta,R. and Kaesberg,P.                        9.86e8
13 173772 Al-Hammad,Y.                                                    4.05e8
14 166392 Abe,M., Ito,N., Sakai,K., Kaku,Y., Oba,M., Nishimura,M., Kuran… 9.91e8
15 150665 Abmayr,S., Gregorevic,P., Allen,J.M. and Chamberlain,J.S.       1.38e9
16  70586 Abdel Razik,E.S.                                                1.94e9
17  51050 Akada,R. and Nakamura,M.                                        9.89e8
18  49795 Alessi,D.R., Saito,Y., Campbell,D.G., Cohen,P., Sithanandam,G.… 1.67e9
19  47644 Agris,P.F., KiKuchi,Y., Gross,H.J., Takano,M. and Sharp,G.C.    2.40e9
20  45633 Abu El-Magd,M.E.R., Abas,H.E., EL-Kattawy,A.M.A. and Mokhbatly… 1.69e9

When you apply dplyr verbs to this database table, they are automatically translated into SQL queries:

R

# Show the SQL queries translated
dat %>%
  dplyr::show_query()

OUTPUT

<SQL>
SELECT `id`, `name`, `crc`
FROM (
  SELECT
    *,
    CASE
WHEN (NOT(((RAND()) IS NULL))) THEN ROW_NUMBER() OVER (PARTITION BY (CASE WHEN (((RAND()) IS NULL)) THEN 1 ELSE 0 END) ORDER BY RAND())
END AS `col01`
  FROM `author`
  WHERE (SUBSTR(`name`, 1, 1) = 'A')
) AS `q01`
WHERE (`col01` <= 20)
ORDER BY `id` DESC

Alternatively, you can use the readepi::read_rdbms() function to import data from a database table. It accepts either an SQL query or a list of query parameters.

4. Extract data from the database

Use dplyr::collect() to force computation of a database query and extract the output to your local computer.

R

# Pull all data down to a local tibble
dat %>%
  dplyr::collect()

OUTPUT

# A tibble: 20 × 3
       id name                                                               crc
    <int> <chr>                                                            <dbl>
 1 342458 An HJ, Koh HM and Song DH.                                      4.11e9
 2 327919 Assumpcao,T.C., Eaton,D.P., Pham,V.M., Francischetti,I.M., Aok… 3.98e9
 3 315865 Anandham,R., Indira Gandhi,P., Kwon,S.W., Sa,T.M., Kim,Y.K. an… 6.86e8
 4 243058 Anderson,O.D., Chao,S., Chin,A., Close,T.J., Crossman,C., Gust… 2.73e9
 5 238726 Atkinson,B.G., Helbing,C.C. and Chen,Y.                         3.93e9
 6 238544 Anderson,M., Amemiya,C., Luer,C., Litman,R., Rast,J., Niimura,… 5.99e8
 7 234381 Al-Zailaie,K.A., Kang,S.W., Youngren,O.M., Thayananuphat,A., B… 3.91e9
 8 216220 Ahsan,M.N., Aoki,H. and Watabe,S.                               3.82e9
 9 197512 Arai,S., Aoki,K., Son,N.T., Tu,V.T., Kikuchi,F., Kinoshita,G.,… 4.18e8
10 181914 Anjanappa,R.B., Gowda,M.M.N., Kanju,E., Nawabu,H., Gruissem,W.… 3.47e9
11 171691 Air,G.M. and Gulati,S.                                          5.99e8
12 157954 Alexander,W.S. and Dunn,A.R.                                    2.75e9
13 151698 Akira,T., Komatsu,M., Nango,R., Tomooka,A., Konaka,K., Yamauch… 4.00e9
14 136764 Adoyo,P.A., Lea,I.A., Richardson,R.T., Widgren,E.E. and O'Rand… 9.73e8
15 120179 Abdelhaleem,M.M.                                                2.02e9
16 102496 Abidi,F., Aissaoui,N., Gaudin,J.C., Chobert,J.M., Haertle,T. a… 2.93e9
17  72480 Anuratha,C.S., Mew,T. and Muthukrishnan,S.                      4.00e9
18  64748 Ashida,H., Tamaki,H., Fujimoto,T., Yamamoto,K. and Kumagai,H.   2.40e9
19  35070 Affolter,M., Montagne,J., Walldorf,U., Groppe,J., Kloter,U., L… 3.93e9
20  15401 Anantharaman,S. and Craft,J.A.                                  2.39e9

Ideally, after specifying a set of queries, we can reduce the size of the input dataset to use in the environment of our R session.

Challenge

Run SQL queries in R using {dbplyr}

Create one table containing:

the column name from table author,
the column acc from table gbCdnaInfo, and
using the author’s id as primary key or common identifier.

Following these steps:

Use dplyr verbs to select column and join tables,
Print the relational database SQL queries, and
Pull out data to your local session.

Give me a hint

Join columns from two different tables:

From the table author, select id and name, keeping only a handful of rows (e.g. authors whose surname starts with "A") — gbCdnaInfo covers every GenBank submission ever made, so narrowing author down first keeps the join fast.
From the table gbCdnaInfo, select author (the foreign key to author$id) and acc.
Join to the table author the table gbCdnaInfo using dplyr::left_join(), matching author$id to gbCdnaInfo$author — note the join columns have different names, so you’ll need by = c("id" = "author") rather than relying on a shared column name.
Print the SQL query using dplyr::show_query()
collect the joined output using dplyr::collect()

Show me the solution

R

# SELECT FEW COLUMNS FROM ONE TABLE AND LEFT JOIN WITH ANOTHER TABLE
author <- rdbms_login %>%
  dplyr::tbl(from = "author") %>%
  dplyr::filter(stringr::str_sub(string = name, start = 1, end = 1) == "A") %>%
  dplyr::slice_sample(n = 5) %>%
  dplyr::select(id, name)

gb_cdna_info <- rdbms_login %>%
  dplyr::tbl(from = "gbCdnaInfo") %>%
  dplyr::select(author, acc)

dplyr::left_join(
  x = author,
  y = gb_cdna_info,
  by = c("id" = "author"),
  keep = TRUE
) %>%
  dplyr::show_query()

OUTPUT

<SQL>
SELECT `LHS`.*, `author`, `acc`
FROM (
  SELECT `id`, `name`
  FROM (
    SELECT
      *,
      CASE
WHEN (NOT(((RAND()) IS NULL))) THEN ROW_NUMBER() OVER (PARTITION BY (CASE WHEN (((RAND()) IS NULL)) THEN 1 ELSE 0 END) ORDER BY RAND())
END AS `col01`
    FROM `author`
    WHERE (SUBSTR(`name`, 1, 1) = 'A')
  ) AS `q01`
  WHERE (`col01` <= 5)
) AS `LHS`
LEFT JOIN `gbCdnaInfo`
  ON (`LHS`.`id` = `gbCdnaInfo`.`author`)

R

dplyr::left_join(
  x = author,
  y = gb_cdna_info,
  by = c("id" = "author"),
  keep = TRUE
) %>%
  dplyr::collect()

OUTPUT

# A tibble: 11 × 4
       id name                                                      author acc
    <int> <chr>                                                      <dbl> <chr>
 1 167020 Anindita,P.D., Sasaki,M., Setiyono,A., Handharyani,E., O… 167020 AB91…
 2 167020 Anindita,P.D., Sasaki,M., Setiyono,A., Handharyani,E., O… 167020 AB91…
 3 167020 Anindita,P.D., Sasaki,M., Setiyono,A., Handharyani,E., O… 167020 AB91…
 4  83972 Alche,J.D., Castro,A.J. and Rodriguez-Garcia,M.I.          83972 AF19…
 5 362706 Aragon E, Wang Q, Zou Y, Morgani SM, Ruiz L, Kaczmarska … 362706 NM_0…
 6 362706 Aragon E, Wang Q, Zou Y, Morgani SM, Ruiz L, Kaczmarska … 362706 NM_0…
 7 362706 Aragon E, Wang Q, Zou Y, Morgani SM, Ruiz L, Kaczmarska … 362706 NM_0…
 8  48298 Aharony,D., Little,J., Thomas,C., Powell,S., Downey-Jone…  48298 S824…
 9  48298 Aharony,D., Little,J., Thomas,C., Powell,S., Downey-Jone…  48298 S762…
10  48298 Aharony,D., Little,J., Thomas,C., Powell,S., Downey-Jone…  48298 NM_0…
11 339309 Alfy,H., Ghreeb,R.Y. and Hafez,E.E.                       339309 MK95…

You can also review the dbplyr R package. But for a step-by-step tutorial about SQL, we recommend you this tutorial about data management with SQL for Ecologist.

We can close the connection to the database with:

R

pool::poolClose(rdbms_login)

Show details

You can confirm the connection closed running the created objects in console:

R

rdbms_login

OUTPUT

<Pool> of MySQLConnection objects
  Objects checked out: 0
  Available in pool: 0
  Max size: Inf
  Valid: FALSE

R

dat

ERROR

Error in `poolCheckout()`:
! The pool has been closed.

Reading from HIS APIs

Health data is increasingly stored in specialized HIS such as Fingertips, GoData, REDCap, DHIS2, SORMAS, etc. The current version of the readepi library allows importing data from DHIS2 and SORMAS. The subsections below demonstrate how to import data from these two systems.

Importing data from DHIS2

DHIS2 (District Health Information System 2) is an open-source software that has revolutionized global health information management. The readepi::read_dhis2() function imports data from the DHIS2 Tracker system via its API.

To successfully import data from DHIS2, you need to:

Connect to the system using the readepi::login() function
Provide the name or ID of the target program and organization unit

You can retrieve the IDs and names of available programs and organization units using the get_programs() and get_organisation_units() functions, respectively.

R

# establish the connection to the system
dhis2_login <- readepi::login(
  type = "dhis2",
  from = "https://play.im.dhis2.org/stable-2-42-6",
  user_name = "admin",
  password = "district"
)

dhis2_login

OUTPUT

<httr2_response>
GET https://play.im.dhis2.org/stable-2-42-6/api/me
Status: 200 OK
Content-Type: application/json
Body: In memory (12468 bytes)

If the step above fails, check for others available in the list of DHIS2 Demo Instances, all accessible with username "admin" and password "district". Just replace stable-2-42-6 in the URL string. The only conditions is that it must be of version equal or lower than 2.42.

Caution

Avoid publishing your USER NAME and PASSWORD. You could use rstudioapi:

R

dhis2_login <- readepi::login(
  type = "dhis2",
  from = "https://play.im.dhis2.org/stable-2-42-6",
  user_name = rstudioapi::askForPassword("Database username"),
  password = rstudioapi::askForPassword("Database password")
)

Your can read further from this blogpost on How to Avoid Publishing Credentials in Your Code

R

# get the names and IDs of the programs
programs <- readepi::get_programs(login = dhis2_login)

# print tables
tibble::as_tibble(programs)

OUTPUT

# A tibble: 15 × 3
   displayName                                         id          type
   <chr>                                               <chr>       <chr>
 1 Acute Viral Hepatitis Case Report — Unified         OhbVrpoiVgR aggregate
 2 Antenatal care visit                                lxAQ7Zs9VYR aggregate
 3 Child Programme                                     IpHINAT79UW tracker
 4 Contraceptives Voucher Program                      kla3mAPgvCH aggregate
 5 Information Campaign                                q04UBOqq3rp aggregate
 6 Inpatient morbidity and mortality                   eBAyeGv0exc aggregate
 7 Malaria case diagnosis, treatment and investigation qDkgAbB5Jlk tracker
 8 Malaria case registration                           VBqh0ynB2wv aggregate
 9 Malaria focus investigation                         M3xtLkYBlKI tracker
10 Malaria testing and surveillance                    bMcwwoVnbSR aggregate
11 MNCH / PNC (Adult Woman)                            uy2gU8kT1jF tracker
12 Provider Follow-up and Support Tool                 fDd25txQckK tracker
13 TB program                                          ur1Edk5Oe2n tracker
14 WHO RMNCH Tracker                                   WSGAb5XwJ3Y tracker
15 XX MAL RDT - Case Registration                      MoUd5BTQ3lY aggregate

R

# get the names and IDs of the organisation units
org_units <- readepi::get_organisation_units(login = dhis2_login)

# print tables
tibble::as_tibble(org_units)

OUTPUT

# A tibble: 1,166 × 8
   National_name National_id District_name District_id Chiefdom_name Chiefdom_id
   <chr>         <chr>       <chr>         <chr>       <chr>         <chr>
 1 Sierra Leone  ImspTQPwCqd Western Area  at6UHUQatSo Rural Wester… qtr8GGlm4gg
 2 Sierra Leone  ImspTQPwCqd Western Area  at6UHUQatSo Rural Wester… qtr8GGlm4gg
 3 Sierra Leone  ImspTQPwCqd Bo            O6uvpzGd5pu Kakua         U6Kr7Gtpidn
 4 Sierra Leone  ImspTQPwCqd Kambia        PMa2VCrupOd Magbema       QywkxFudXrC
 5 Sierra Leone  ImspTQPwCqd Tonkolili     eIQbndfxQMb Yoni          NNE0YMCDZkO
 6 Sierra Leone  ImspTQPwCqd Port Loko     TEQlaapDQoK Kaffu Bullom  vn9KJsLyP5f
 7 Sierra Leone  ImspTQPwCqd Koinadugu     qhqAxPSTUXp Nieni         J4GiUImJZoE
 8 Sierra Leone  ImspTQPwCqd Western Area  at6UHUQatSo Freetown      C9uduqDZr9d
 9 Sierra Leone  ImspTQPwCqd Western Area  at6UHUQatSo Freetown      C9uduqDZr9d
10 Sierra Leone  ImspTQPwCqd Kono          Vth0fbpFcsO Gbense        TQkG0sX9nca
# ℹ 1,156 more rows
# ℹ 2 more variables: Facility_name <chr>, Facility_id <chr>

After retrieving organization units and program names from the DHIS2 database, we can import data using either names or coded IDs, as demonstrated in the code chunks below:

R

# import data from DHIS2 using names
data_name <- readepi::read_dhis2(
  login = dhis2_login,
  org_unit = "Bucksal Clinic",
  program = "Child Programme"
)

tibble::as_tibble(data_name)

OUTPUT

# A tibble: 30 × 26
   event      tracked_entity org_unit Gender `First name` `Last name` enrollment
   <chr>      <chr>          <chr>    <chr>  <chr>        <chr>       <chr>
 1 RrWEjrd84… yzhEctxhPiL    Bucksal… Female Karen        Alvarez     WKgHJZ3Ue…
 2 JgPqmTcG0… G3hZ9gN7UYD    Bucksal… Female Ruby         Warren      Rth5aVYua…
 3 Sz2U8t3YA… G3hZ9gN7UYD    Bucksal… Female Ruby         Warren      Rth5aVYua…
 4 VEvcoYpWF… RyPuD70zgE9    Bucksal… Male   Earl         Mason       COU4sScB6…
 5 BNZA0qyfC… KfXae2GB6Fb    Bucksal… Male   Mark         Jacobs      x4vAlqBJl…
 6 wGMKQ3SBb… KfXae2GB6Fb    Bucksal… Male   Mark         Jacobs      x4vAlqBJl…
 7 FoCWOlstb… aXaALEYwQNV    Bucksal… Female Lillian      Mccoy       VkZrYFMCK…
 8 HFQQUGE9O… aXaALEYwQNV    Bucksal… Female Lillian      Mccoy       VkZrYFMCK…
 9 pVmIV0EyY… rdo8mO4Jifk    Bucksal… Female Denise       Henderson   iwYMBJgiQ…
10 Dee74ydRn… rdo8mO4Jifk    Bucksal… Female Denise       Henderson   iwYMBJgiQ…
# ℹ 20 more rows
# ℹ 19 more variables: program <chr>, program_stage <chr>, event_date <chr>,
#   `MCH Infant Feeding` <chr>, `MCH OPV dose` <chr>, `MCH BCG dose` <chr>,
#   `MCH ARV at birth` <chr>, `MCH Apgar Score` <chr>, `MCH Weight (g)` <chr>,
#   `MCH Infant Weight  (g)` <chr>, `MCH Vit A` <chr>,
#   `MCH Infant HIV Test Result` <chr>, `MCH HIV Test Type` <chr>,
#   `MCH IPT dose` <chr>, `MCH DPT dose` <chr>, `MCH Child ARVs` <chr>, …

R

# import data from DHIS2 using IDs
data_id <- readepi::read_dhis2(
  login = dhis2_login,
  org_unit = "vRC0stJ5y9Q",
  program = "IpHINAT79UW"
)

identical(data_id, data_name)

OUTPUT

[1] TRUE

Note that not all organization units are registered for a specific program. To find which organization units are running a particular program, use the get_program_org_units() function as shown below:

R

# get the list of organisation units that run the program "IpHINAT79UW"
target_org_units <- readepi::get_program_org_units(
  login = dhis2_login,
  program = "IpHINAT79UW",
  org_units = org_units
)

tibble::as_tibble(target_org_units)

OUTPUT

# A tibble: 1,166 × 3
   org_unit_ids levels        org_unit_names
   <chr>        <chr>         <chr>
 1 vRC0stJ5y9Q  Facility_name Bucksal Clinic
 2 simyC07XwnS  Facility_name Maforay MCHP
 3 E9oBVjyEaCe  Facility_name Gbanja Town MCHP
 4 ZpE2POxvl9P  Facility_name Faabu CHP
 5 yTMrs5kClCv  Facility_name Condama MCHP
 6 FO1Tq8vUa62  Facility_name EPI Headquarter
 7 jGYT5U5qJP6  Facility_name Gbaiima CHC
 8 LaxJ6CD2DHq  Facility_name EM&BEE Maternity Home Clinic
 9 WerHl8SDtRU  Facility_name Mandema CHP
10 CTnuuI55SOj  Facility_name Manewa MCHP
# ℹ 1,156 more rows

Challenge

Reading from a DHIS2 sever

Test readepi by accessing to a DHIS2 server with your credentials.

Do the following:

Log into a different server,
List all available programs and organization units,
Read data from one of these programs,
Optional: Reproduce one descriptive figure.

Give me a hint

Try using rstudioapi::askForPassword() for user_name and password.

If you get errors, please fill an issue in the readepi GitHub repository.

Importing data from SORMAS

The SORMAS (Surveillance Outbreak Response Management and Analysis System) is an open-source e-health system that optimizes infectious disease surveillance and outbreak response processes. The readepi::read_sormas() function allows you to import data from SORMAS via its API.

In the current version of the readepi package, the read_sormas() function returns data for the following columns: case_id, person_id, sex, date_of_birth, case_origin, country, city, lat, long, case_status, date_onset, date_admission, date_last_contact, date_first_contact, outcome, date_outcome, and Ct_values.

The code chunk below demonstrates how to import data from a demo SORMAS system:

R

# CONNECT TO THE SORMAS SYSTEM
sormas_login <- readepi::login(
  type = "sormas",
  from = "https://demo.sormas.org/sormas-rest",
  user_name = "SurvSup",
  password = "Lk5R7JXeZSEc"
)

# FETCH ALL COVID (Corona virus) CASES FROM THE TEST SORMAS INSTANCE
covid_cases <- readepi::read_sormas(
  login = sormas_login,
  disease = "coronavirus",
)
tibble::as_tibble(covid_cases)

OUTPUT

# A tibble: 2 × 15
  case_id             person_id date_onset case_origin case_status outcome sex
  <chr>               <chr>     <date>     <chr>       <chr>       <chr>   <chr>
1 TYG7I3-2GONYO-JDJP… QJXOUU-6… NA         IN_COUNTRY  SUSPECT     NO_OUT… <NA>
2 ULMPMT-PBQOQ2-ETGY… WVP6NB-J… 2026-05-31 IN_COUNTRY  NOT_CLASSI… NO_OUT… <NA>
# ℹ 8 more variables: date_of_birth <chr>, country <chr>, city <chr>,
#   latitude <chr>, longitude <chr>, contact_id <chr>,
#   date_last_contact <date>, Ct_values <chr>

A key parameter is the disease name. To ensure correct syntax, you can retrieve the list of available disease names using the sormas_get_diseases() function.

R

# get the list of all disease names
disease_names <- readepi::sormas_get_diseases(
  login = sormas_login
)

tibble::as_tibble(disease_names)

OUTPUT

# A tibble: 69 × 2
   disease            active
   <chr>              <chr>
 1 AFP                TRUE
 2 CHOLERA            TRUE
 3 CONGENITAL_RUBELLA TRUE
 4 DENGUE             TRUE
 5 EVD                TRUE
 6 GUINEA_WORM        TRUE
 7 LASSA              TRUE
 8 MONKEYPOX          TRUE
 9 NEW_INFLUENZA      TRUE
10 PLAGUE             TRUE
# ℹ 59 more rows

Challenge

Reading from Demo SORMAS sever

Log into this server, list all available diseases, and import cases related to the monkeypox (mpox) disease.

Show me the solution

R

# establish the connection to the system
sormas_demo <- readepi::login(
  type = "sormas",
  from = "https://demo.sormas.org/sormas-rest",
  user_name = "SurvSup",
  password = "Lk5R7JXeZSEc"
)

# List the names of all disease
demo_diseases <- readepi::sormas_get_diseases(login = sormas_demo)
tibble::as_tibble(demo_diseases)

OUTPUT

# A tibble: 69 × 2
   disease            active
   <chr>              <chr>
 1 AFP                TRUE
 2 CHOLERA            TRUE
 3 CONGENITAL_RUBELLA TRUE
 4 DENGUE             TRUE
 5 EVD                TRUE
 6 GUINEA_WORM        TRUE
 7 LASSA              TRUE
 8 MONKEYPOX          TRUE
 9 NEW_INFLUENZA      TRUE
10 PLAGUE             TRUE
# ℹ 59 more rows

R

# get the list of all disease names
mpox_cases <- readepi::read_sormas(
  login = sormas_demo,
  disease = "monkeypox",
)

WARNING

Warning in as.POSIXct(as.numeric(x), origin = "1970-01-01"): NAs introduced by
coercion

R

tibble::as_tibble(mpox_cases)

OUTPUT

# A tibble: 1 × 15
  case_id             person_id date_onset case_origin case_status outcome sex
  <chr>               <chr>     <date>     <chr>       <chr>       <chr>   <chr>
1 TU75PT-MMBE3I-OEJN… XQFYXH-R… NA         IN_COUNTRY  NOT_CLASSI… NO_OUT… <NA>
# ℹ 8 more variables: date_of_birth <chr>, country <chr>, city <chr>,
#   latitude <chr>, longitude <chr>, contact_id <chr>,
#   date_last_contact <date>, Ct_values <chr>

Key Points

Use rio, io, readr or {ImportExport} to read data from individual files.
Use readepi to read data from RDBMS and HIS.
The {rio} package supports a wide range of file formats including CSV, TSV, XLSX, and compressed files.
Use readepi::login() to establish connections to RDBMS, DHIS2, or SORMAS systems.
The readepi package currently supports importing data from DHIS2 and SORMAS health information systems.

Read case data

Overview

Questions

Objectives

Prerequisites

Introduction

R

The double-colon (::) operator

Setup a project and folder

Reading from files

R

OUTPUT

Why should we use the {here} package?

Reading compressed data

Give me a hint

R

Show me the solution

R

Reading from databases

Advantages of reading data directly from a database?

1. Connect with a database

R

OUTPUT

R

OUTPUT

2. Access the list of tables from the database

R

OUTPUT

3. Read data from a table in a database

R

OUTPUT

R

OUTPUT

4. Extract data from the database

R

OUTPUT

Run SQL queries in R using {dbplyr}

Give me a hint

Show me the solution

R

OUTPUT

R

OUTPUT

R

Show details

R

OUTPUT

R

ERROR

Reading from HIS APIs

Importing data from DHIS2

R

OUTPUT

R

R

OUTPUT

R

OUTPUT

R

OUTPUT

R

OUTPUT

R

OUTPUT

Reading from a DHIS2 sever

Give me a hint

Importing data from SORMAS

R

OUTPUT

R

OUTPUT

Reading from Demo SORMAS sever

Show me the solution

R

OUTPUT

R

WARNING

R

OUTPUT

The double-colon (`::`) operator