10 Working effectively with data

 

This chapter covers

  • Best practices for working with data
  • Serialization
  • Type providers
  • Data visualization
  • Tooling

When I first started using F#, coming from a C# background and used to writing database-driven web applications and the like, I was amazed that I could use .NET for analytical workloads, ad hoc data exploration, and even machine learning. This chapter will give you an overview of some of the different tools and techniques you can use to effectively start working with external data sources within F#, both for full-blown applications and for ad hoc analysis.

10.1 Best practices

Working with data in F# is something you’ll be doing all the time, regardless of the source (e.g., a SQL database, web service, an event stream, etc.). This section covers some general best practices for working with collections of data in general.

10.1.1 Data is data

One of the things I like about F# is that there is a common set of principles and practices that you can apply across all data, no matter where it comes from—whether it’s loading a JSON file with stock price data, data that’s come from a SQL database with order information, transient data that represents a different shape of customer information that you’re dealing with in a pipeline, or data that is sourced from user input. All of these represent some form of data and can normally be treated in the same way.

10.1.2 Separation of data and functionality

10.1.3 Don’t treat scripts as programs

10.1.4 Working with lists of results

10.2 Serialization of data

10.2.1 Deserializing JSON to F#

10.2.2 Acceptable alternatives to DTOs

10.2.3 Serializing F# into JSON

10.3 Type providers

10.3.1 Working with the JSON type provider

10.3.2 Working with external data sources

10.3.3 More data types from FSharp.Data