What you will learn in The Well-Grounded Data Analyst
Solve messy data problems like a pro
Chapter by chapter
Chapter 1: Bridging the gap between data science training and the real world
What you learn in chapter 1 →Concepts explained in this book
- missing valueA missing value is an absent or unavailable value for a field that is expected to contain data.
- granularityThe level of detail at which data or an analysis is represented, such as cities, boroughs, or broader regions.
- unique identifierA unique identifier is a value intended to distinguish one entity or record from others within a data source or model.
- thresholdA cutoff is a threshold that determines whether an observation qualifies for inclusion in an analysis.
- categorical dataCategorical data consists of values representing membership in discrete, named groups or categories.
- deduplicationDeduplication is the process of identifying and handling multiple records that represent the same underlying entity.
- data aggregationAggregation combines records into summary values, often after grouping them by one or more categories.
- placeholderA placeholder value is a deliberately assigned substitute representing a source value that cannot be directly encoded in the required data format.
- entity resolutionEntity resolution is the process of determining which records refer to the same real-world entity, including through deduplication and record linkage.
- time seriesA time series is a sequence of observations indexed by time, typically representing measurements collected across successive time points.
- annual aggregationAnnual aggregation is the process of summarizing observations into values representing individual calendar years.
- quartileA quartile is one of the values dividing an ordered dataset into four equal parts.
- transformThe pandas transform method applies a function to grouped data and returns results aligned with the original data shape.
- window functionA window function is a SQL operation that computes a result over related rows while retaining each row in the query result.
- data dictionaryA data dictionary is structured documentation describing a dataset's fields, meanings, formats, and permissible values.
- results-driven approachA results-driven approach is an analytical or problem-solving workflow that prioritizes useful outcome-oriented results and refines work as needed.
- indicator variableAn indicator variable is a variable that encodes whether a condition or category applies, typically with binary values such as zero and one.
- sanity checkA sanity check is a basic validation intended to detect obvious inconsistencies or implausible results in data or analysis.
- schemaA schema is a specification of the fields, structure, and expected organization of data in a dataset or table.
- provenanceData lineage is the traceable record of data's origins, transformations, and movement across sources and systems.
- session durationSession duration is the elapsed time between the beginning and end of a browsing session.
- anomalyAn outlier is an observation whose value differs substantially from the typical pattern of the other observations.
- Likert scaleLikert data consists of ordered response categories that measure the degree of agreement, favorability, or another attitude.
- data modelA formal representation of the entities, attributes, relationships, and constraints in a data domain.
- completenessCompleteness is the extent to which the expected records or attributes in a dataset are present and available.
- conversionA conversion rate is the percentage of opportunities or interactions that result in a desired outcome, such as a product purchase after cart placement.
- clusteringClustering is the process of grouping observations so that members of the same group are more similar to one another than to members of other groups.
- volumeSales volume is the quantity of units of a product sold during a specified period.
- pandaspandas is a Python software library for working with tabular and labeled data, including data manipulation and analysis.
- one-hot encodingOne-hot encoding transforms categorical values into separate binary indicator variables, with each variable representing one category.
- long dataLong data is a tabular representation in which each row records one measurement or entity-participation observation.
- count pointA count point is a physical measurement location where traffic observations are collected.
- composite keyA composite primary key is a record key whose uniqueness is determined by a combination of multiple columns.
- left outer joinA left join combines records from two datasets while retaining every record from the left dataset and matching records from the right dataset when available.
- seabornSeaborn is a Python visualization library that provides a high-level interface for creating statistical graphics.
- minimum viable answerA minimum viable answer is an initial answer that is sufficient to provide immediate value and serve as a basis for further discussion or iteration.
- colormapA colormap is a mapping from data values to a range or set of visual colors.
- heat mapA heatmap is a visualization that encodes values in a matrix or table with varying color intensity.
- str.get_dummiesstr.get_dummies is a pandas string method that expands delimited categorical values into binary indicator columns.
- data analysisData analysis is the systematic process of inspecting, transforming, and interpreting data to produce findings or support decisions.
- genreA film genre is a category of films defined by shared themes, narrative conventions, or stylistic characteristics.
- probability distributionA statistical distribution describes the pattern of frequencies or probabilities across a variable's possible values.
- cinema admissionA cinema admission is a recorded instance or count of a person attending a film screening.
- app categoryAn app category is a broader class assigned to mobile applications according to their function or subject.
- data cleaningData cleaning is the process of detecting, correcting, standardizing, or removing inaccurate, inconsistent, incomplete, or irrelevant data.
- distributorA film distributor is an entity responsible for bringing films to exhibitors, platforms, or audiences through release and licensing activities.
- ordinal dataOrdinal data consists of categorical values whose categories have a meaningful natural order.
- StreamlitStreamlit is an open-source Python library for building interactive data applications with minimal application code.
- browsing sessionA browsing session is a period during which a user continuously interacts with a website or application.
- session countSession count is the number of recorded browsing sessions associated with a user or other entity.
- setA set is an unordered collection of distinct elements that supports membership testing and set operations.
- domain knowledgeDomain expertise is specialized knowledge of the subject area represented by the data or analysis.
- Python Record Linkage ToolkitThe Python Record Linkage Toolkit, also known as recordlinkage, is a Python library for comparing, deduplicating, and linking records.
- market shareMarket share is the proportion of total activity, sales, or revenue attributable to a product, organization, or category within a defined market.
- dropdownA Streamlit input widget that displays selectable options in a dropdown control.
- league tableA league table is a ranked tabular presentation of entities ordered by a selected metric or score.
- Damerau–Levenshtein distanceDamerau–Levenshtein distance is an edit-distance measure that counts character insertions, deletions, substitutions, and transpositions needed to transform one string into another.
- record linkageRecord linkage is the process of matching records across or within datasets to determine whether they describe the same entity.
- research questionA focused question that an investigation is intended to answer using evidence.
- revenueRevenue is the income generated from an organization's sales of goods or services before applicable deductions or expenses.
- inertiaInertia is a clustering metric that quantifies within-cluster variation, typically as the sum of squared distances from observations to their assigned cluster centers.
- event funnelAn event funnel is a sequence of behavioral stages in which progressively fewer users or events reach later stages.
- ParquetParquet is a compressed columnar file format that preserves data types and is designed for efficient analytical storage and retrieval.
- sessionA session is a period of a user's interaction with applications represented by related event records.
- session identifierA session identifier is a value assigned to distinguish one user session from another within event data.
- exact duplicateAn exact duplicate is a record whose relevant field values are identical to those of another record.
- tsv formatTSV is a plain-text tabular data format in which fields are separated by tab characters and records are represented on separate lines.
- pivot tableA pivot table is a summarized tabular view that groups and aggregates records across selected row and column dimensions.
- month-over-month comparisonMonth-on-month change is the difference in a measure between one month and the preceding month.
- percentage changePercentage change is the relative increase or decrease between an initial value and a later value, expressed as a percentage.
- temporal patternA temporal pattern is a recurring or otherwise meaningful structure in data that is associated with time.
- elbow methodThe elbow method selects an approximate number of clusters by identifying where additional clusters produce diminishing improvement in a clustering criterion.
- K-means clusteringK-means is a clustering algorithm that partitions observations into a specified number of groups by minimizing distances from observations to their assigned cluster centers.
- z-scoreA z-score is a standardized value expressing how many standard deviations an observation lies above or below a variable's mean.
- covid periodA COVID period is a time classification distinguishing conditions before, during, or after COVID-19 lockdowns.
- gross box officeGross box office is the total ticket revenue generated by films before deductions or distribution of proceeds.
- freeholdA freehold is a property tenure in which the owner holds the property and the land it occupies indefinitely.
- leaseholdA leasehold is a property tenure in which rights to occupy or use a property and its structure are held for a defined lease period, usually subject to fees.
- new buildA new build is a property newly constructed at or near the time of its recorded purchase.
- nonstandard transactionA nonstandard transaction is a property sale or transfer that does not represent a standard property purchase.
- pdf text extractionPDF extraction is the process of retrieving structured or unstructured content from PDF documents.
- user surveyA survey is a structured data-collection instrument that gathers responses to predefined questions from participants.
- data layerA data access layer is a software layer that reads, retrieves, and prepares data for other application components.
- data manipulation routineData manipulation is the process of sorting, grouping, reshaping, or otherwise transforming data to support analysis.
- property marketA property market is the system of buying, selling, and pricing real estate within a geographic area.
- PDFPDF is a standardized document format that preserves the visual layout of text, graphics, and tables across systems.
- raw dataRaw data is data in its original, unprocessed form before business logic or analytical transformations are applied.
- cycling infrastructureCycling infrastructure comprises physical facilities and design features that support bicycle travel, such as cycle lanes, paths, and crossings.
- table extractionThe process of recovering tabular rows and columns from a document or other source.
- minimum viable data modelA minimum viable data model is an initial data model that provides useful analytical capability with only the essential structure and processing.
- visualizationData visualization is the graphical representation of data to reveal patterns, comparisons, or trends.
- market gapA market gap is an unmet customer need or underserved segment that may provide an opportunity for a product or service.
- statistical significanceStatistical significance is the degree to which an observed result is unlikely under a specified null hypothesis according to a statistical test.
- ridgeline plotA ridgeline plot is a visualization that displays multiple distributions as overlapping, vertically offset density or frequency plots.
- hypothesisA hypothesis is a specific, testable proposition about a relationship or difference in a domain.
- stakeholderA stakeholder question is a question posed by a person or group affected by, responsible for, or interested in an analytical outcome.
- varianceVariance is a statistical measure of the average squared deviation of observations from their mean.
- st.cache_dataA Streamlit decorator that caches data-returning function results to avoid repeated computation or loading.
- continuous dataContinuous data consists of numerical measurements that can take values across a range, including arbitrarily fine subdivisions.
- use caseA use case is a specific situation or purpose for which a tool, system, or method is applied.
- trendA trend is a general direction of change or recurring development in observations over time or across conditions.
- wide formatWide data is a tabular representation in which each entity occupies a row and measurements or attributes occupy columns.
- forecasting algorithmA forecasting algorithm is a computational procedure that predicts future values from historical or other explanatory data.
- ubiquitous languageBusiness terminology is the agreed meanings of domain-specific terms used by people involved in an analysis.
- geographic breakdownAn organization of data according to geographic areas or levels of geographic detail.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.