A programming environment is the workspace where software or data-pipeline code is created, modified, prototyped, unit-tested, and optimized before formal testing or production deployment. It may be a local computer setup or a shared development environment. section 6.3 section 15.1
Prepare a local environment for Snowpark Python development:
- Install Python locally, using a version supported by Snowflake. section 6.3.1
- Create a virtual environment with conda or venv. section 6.3.1
- Install the snowflake-snowpark-python package with conda or pip. section 6.3.1
- Use an IDE such as Visual Studio Code, configured to use the supported Python version. section 6.3.1
Use the development environment to build prototypes, compare implementation options, maintain pipeline code, and share changes through a centralized repository. Deploy completed code to test, user acceptance, and production environments according to organizational needs. section 15.0 section 15.1
Table 15.1 Types of data testing and the data they require
|
Test type
|
Data requirements
|
|---|---|
|
Data pipeline functionality
|
Small data sample
|
|
Data pipeline performance
|
Data volume similar to the production data volume
|
|
Data integrity
|
Data that closely resembles the data from the production environment
|
|
Data governance features
|
When production data contains sensitive information that should not be shared in the development and test environments, generated data or masked data can be used for testing.
|
Choose test data according to the validation goal: samples for functionality, production-scale volumes for performance, representative data for integrity, and masked or generated data for governance testing. section 15.1
