Identify the challenges associated with processing data.
DAT-2.C.1 The ability to process data depends on the capabilities of the users and their tools.
DAT-2.C.2 Data sets pose challenges regardless of size, such as:
- the need to clean data
- incomplete data
- invalid data
- the need to combine data sources
DAT-2.C.3 Depending on how data were collected, they may not be uniform. For example, if users enter data into an open field, the way they choose to abbreviate, spell, or capitalize something may vary from user to user.
DAT-2.C.4 Cleaning data is a process that makes the data uniform without changing their meaning (e.g., replacing all equivalent abbreviations, spellings, and capitalizations with the same word).
DAT-2.C.5 Problems of bias are often created by the type or source of data being collected. Bias is not eliminated by simply collecting more data.
DAT-2.C.6 The size of a data set affects the amount of information that can be extracted from it.
DAT-2.C.7 Large data sets are difficult to process using a single computer and may require parallel systems.
DAT-2.C.8 Scalability of systems is an important consideration when working with data sets, as the computational capacity of a system affects how data sets can be processed and stored.