Inputs and Outputs in BharatSim
Reading inputs from a synthetic population
BharatSim uses a CSV file as an input. It is equipped to ingest data from a file, by reading it and converting the data into the Graph used by the framework.
In order to ingest data, we need to make use of three functions:
ingestData
: which is a method of theSimulation
classingestCSVData
: a method of theContextBuilder
objectA user-defined “mapper” function which tells the program what data to extract and what to do with it
Using the framework-defined functions
First, we need to import the necessary packages:
import com.bharatsim.engine.ContextBuilder._
import com.bharatsim.engine.execution.Simulation
import com.bharatsim.engine.graph.ingestion.{GraphData, Relation}
Hint
You’ll see why we import GraphData
and Relation
in the next section!
The next step is to create an instance of the simulation class in the main function,
val simulation = Simulation()
We then ingest the data in the following way:
simulation.ingestData(implicit context => {
ingestCSVData("input.csv", myCsvDataExtractor)
logger.debug("Ingestion done")
})
where myCsvDataExtractor
is the user-defined “mapper” function.
Note
The above block of code essentially causes the data from the CSV file to be read one line at a time
Creating the user-defined mapping function
The user-defined function (myCsvDataExtractor
, in our case) will depend on the data we want to extract. As an example, let’s consider that we have data on a number of cats, each with their own ID, name, city of residence, an integer ID for the city, and a particular colour. Our CSV file would look like
ID |
Name |
City |
CityID |
Colour |
---|---|---|---|---|
0 |
Mittens |
Sydney |
2000 |
White |
1 |
Tabby |
London |
1050 |
Brown |
2 |
Garfield |
LasagnaLand |
7 |
Orange |
3 |
Elizabeth |
Mishelam |
102 |
Black |
4 |
Coppe |
Crossbell |
100 |
Black |
5 |
Marie |
Crossbell |
100 |
Orange |
6 |
Antoine |
Zeiss |
62 |
Brown |
Let’s assume we’ve already defined the following:
a case class
Cat
with three attributes,id
,name
andcolour
another case class
City
with two attributes,id
andcityname
Our function myCsvDataExtractor
should do the following
accept a map of keys (the CSV headers) and values (the CSV element in the row corresponding to the header)
accept the context as an implicit parameter
return a
GraphData
object (which you can read about here)
Note
The map is already provided by the ingestCsvData
function
private def myCsvDataExtractor(map: Map[String, String](implicit context: Context): GraphData = {}
The first thing we need to do in the function is store the CSV data to appropriate variables.
val catName = map("Name").toString
val catID = map("ID").toLong
val catCity = map("City").toString
val catCityID = map("CityID").toLong
val catColour = map("Colour").toString
Note
The key of the map
is the header from the CSV file.
We then use a Constructor to create an instance of the Cat
class, for the cat pertaining to a particular row in the CSV. We then do the same for the City
class.
val singleCat: Cat = Cat(
catID,
catName,
catColour
)
val singleCity: City = City(
catCityId,
catCity
)
Next, we establish relations that will link nodes on the graph. We make a livesIn
relation between the cat and the city, and a contains
relation between the city and the cat. To do this, we specify the classes the relation is formed between, and then the unique IDs of the nodes with the relation in between them.
val livesIn = Relation[Cat, City](catID, "LIVES_IN", catCityID)
val contains = Relation[City, Cat](catCityID, "CONTAINS", catID)
We then create an instance of the GraphData
class, and add the nodes and relations to it
val graphData = GraphData()
graphData.addNode(catID, singleCat)
graphData.addNode(catCityID, singleCity)
graphData.addRelations(staysAt, contains)
Note
The first parameter of graphData.addNode
is the unique key of the node.
Finally, we need our function to return the graphData
object we’ve made:
graphData
Hint
In Scala, the last line of a function is treated as a return, and so this is valid syntax.
Putting it all together, our user-defined myCsvDataExtractor
function is
private def myCsvDataExtractor(map: Map[String, String])(implicit context: Context): GraphData = {
val catName = map("Name").toString
val catID = map("ID").toLong
val catCity = map("City").toString
val catCityID = map("CityID").toLong
val catColour = map("Colour").toString
val singleCat: Cat = Cat(
catID,
catName,
catColour
)
val singleCity: City = City(
catCityId,
catCity
)
val livesIn = Relation[Cat, City](catID, "LIVES_IN", catCityID)
val contains = Relation[City, Cat](catCityID, "CONTAINS", catID)
val graphData = GraphData()
graphData.addNode(catID, singleCat)
graphData.addNode(catCityID, singleCity)
graphData.addRelations(staysAt, contains)
graphData
}
Note
You may have noticed that in the CSV file, two cats (namely, Coppe and Marie) both live in the same city (Crossbell). That does not, however, lead to two nodes being created for the same city. A node is defined by it’s unique key and it’s instance. In this example, the unique key is the city ID (which is the same for both cats - 100
) and the instance is the corresponding object singleCity
, which is again identical for both the cats (the attributes are 100
and "Crossbell"
, respectively). As such, the same node is used, and the city doesn’t duplicate in the graph.
Writing outputs to a CSV file
A convenient way to store the output is by using a CSV file. Scala is capable of writing to files, but BharatSim simplifies the process when it comes to CSV outputs.
Note
In case the quantities you’d like to output are fairly simple, you could use Scala’s println
function to directly output what you need.
Saving your output to a CSV file
BharatSim relies on a trait called SimulationListener
to help output data .
SimulationListener
contains 4 methods, each of which allow us to perform a task in one of the following situations:
At the start of the simulation
At the start of every time step
At the end of every time step
At the end of the simulation
The BharatSim engine also contains a class called CsvOutputGenerator
, an extension of SimulationListener
which has two attributes:
path
, the desired path for the output file to be storedcsvSpecs
, a user-defined class that outputs the headers and the rows required. Note that this user-defined class should extend theCSVSpecs
trait and override thegetHeaders
andgetRows
methods.
This class writes the headers at the start of the simulation, writes the rows at the start of every time step, and closes the writer at the end of the simulation.
Output at a single instant of time
We can define a class as follows:
import com.bharatsim.engine.Context
import com.bharatsim.engine.listeners.CSVSpecs
class MyOutputSpec(context: Context) extends CSVSpecs {
override def getHeaders: List[String] =
List(
"Header1",
"Header2",
"Header3"
)
override def getRows(): List[List[Any]] = {
val elementInRow: String = "row" + context.getCurrentStep.toString
val row = List(
elementInRow,
elementInRow,
elementInRow
)
List(row)
}
}
Now, we need to create an instance of the CsvOutputGenerator
class that uses MyOutputSpec
, and call the required methods. First, we need to import CsvOutputGenerator
into our main class:
import com.bharatsim.engine.listeners.CsvOutputGenerator
Next, we add the following code snippet inside simulation.defineSimulation
in the main function:
var outputGenerator = new CsvOutputGenerator("src/main/resources/output.csv", new MyOutputSpec(context))
outputGenerator.onSimulationStart(context)
outputGenerator.onStepStart(context)
outputGenerator.onSimulationEnd(context)
Note
Calling the onStepEnd
method of the class isn’t necessary, as the CsvOutputGenerator
class currently does nothing when it’s called.
The output is
Header1 |
Header2 |
Header3 |
---|---|---|
row0 |
row0 |
row0 |
Hint
In case you want your outputs generated after the simulation is completed, you can place the above 4 lines of code inside simulation.onCompleteSimulation
.
You can see a more in-depth example of this in Saving location-level information from the simulation.
Output at every time step
If we’d like to investigate the dynamics of the simulation as it evolves with time, we essentially need to call the three methods described above every time step. BharatSim simplifies things with SimulationListenerRegistry
, which allows us to register the output generator in the simulation (similar to how we registered agents), so that it writes data to the CSV file at every time step.
First, we must import CsvOutputGenerator
and SimulationListenerRegistry
import com.bharatsim.engine.listeners.{CsvOutputGenerator, SimulationListenerRegistry}
Next, we register it using the register
method of SimulationListenerRegistry
. Note that the following code snippet must go inside simulation.defineSimulation
in the main function.
SimulationListenerRegistry.register(
new CsvOutputGenerator("src/main/resources/output.csv", new myOutputSpec(context))
)
where myCsvSpecs
is the user-defined class which requires the context as an attribute.
Now, the output is
Header1 |
Header2 |
Header3 |
---|---|---|
row1 |
row1 |
row1 |
row2 |
row2 |
row2 |
row3 |
row3 |
row3 |
row4 |
row4 |
row4 |
row5 |
row5 |
row5 |
and so on, until the tick at which the simulation ends.
Hint
Running the above block of code once will cause a file called output
to be created at src/main/resources/
. However, running it again will rewrite the contents of the file with the new output. You can get around this by adding the current time to the output as a string. For example,
val currentTime = new Date().getTime
SimulationListenerRegistry.register(
new CsvOutputGenerator("src/main/resources/output_" + currentTime + ".csv", new SIROutputSpec(context))
)
Note that Date().getTime
returns the time as a UNIX timestamp, and so your output will contain a long integer after the underscore.
For a more detailed example of how to output data to a CSV file, please refer to the Writing your first program section.