Data Sources

Configure connections to external databases and data sources for seamless integration with your existing data infrastructure.

For connecting to your Snowflake data warehouse:

1. Go to Settings → Data Sources
2. Add New Connection
3. Enter your database details:
   • Account Identifier: Your Snowflake account name
   • Username: Your Snowflake username
   • Password: Your Snowflake password (securely encrypted)
   • Warehouse: Default warehouse for queries
   • Database: Default database to connect to
   • Schema: Default schema within the database
4. Test Connection to verify credentials
5. Save for future use

For PostgreSQL database connections:

Required Information

• Host: Database server address
• Port: Database port (typically 5432)
• Database: Database name
• Username: Database username
• Password: Database password

Optional Settings

• SSL Mode: Connection security settings
• Connection Timeout: Query timeout settings
• Schema: Default schema to use

Account: your-company.snowflakecomputing.com
Username: your_username
Password: your_secure_password
Warehouse: COMPUTE_WH
Database: ANALYTICS_DB
Schema: PUBLIC
Role: ANALYST_ROLE

Host: analytics-db.company.com
Port: 5432
Database: analytics
Username: analyst
Password: secure_password
SSL Mode: require

Host: localhost
Port: 5432
Database: local_analytics
Username: postgres
Password: local_password
SSL Mode: prefer

Local Processing

• All queries execute directly against your database
• No data is transmitted to external servers
• Results are processed entirely on your local machine
• Full compliance with data privacy regulations

Credential Security

• Passwords encrypted using industry-standard encryption
• Credentials stored only on your local machine
• Support for password-less authentication methods
• Integration with enterprise identity providers

Connection Security

• SSL/TLS encryption for all database connections
• Support for VPN and private network connections
• Firewall-friendly connection pooling
• Certificate-based authentication where supported

Access Control

• Database-level permissions respected
• Row-level security support
• Schema and table access controls
• Audit logging of database queries

Organization Features

• Connection Names: Give descriptive names to each connection
• Connection Groups: Organize connections by environment or purpose
• Default Connections: Set preferred connections for different data types
• Connection Status: Monitor connection health and availability

Environment Management

• Development: Connect to development databases for testing
• Staging: Use staging environments for validation
• Production: Secure connections to production data
• Backup: Configure backup database connections

Verification Process

• Test Connection: Verify credentials and network connectivity
• Schema Browsing: Preview available tables and schemas
• Query Testing: Run test queries to validate permissions
• Performance Testing: Check query response times

Health Monitoring

• Connection Status: Real-time connection health indicators
• Query Performance: Monitor query execution times
• Error Tracking: Track and resolve connection issues
• Usage Statistics: Monitor database usage patterns

Query Optimization

• Automatic query pushdown for better performance
• Intelligent sampling for large tables
• Parallel query execution when possible
• Automatic caching of frequently accessed data

Resource Management

• Configurable warehouse size recommendations (Snowflake)
• Query cost estimation before execution
• Automatic query cancellation for runaway operations
• Connection pooling for multiple concurrent analyses

Data Source Discovery

• Automatic Schema Detection: Discover tables and relationships
• Metadata Import: Import column descriptions and data types
• Data Lineage: Track data source dependencies
• Change Detection: Monitor schema and data changes

Synchronization Options

• Real-Time Updates: Live connection to changing data
• Scheduled Refresh: Periodic data synchronization
• Incremental Loading: Efficient updates for large datasets
• Change Data Capture: Track and process data changes

Supported Formats

• CSV: Automatic delimiter and encoding detection
• Parquet: High-performance columnar format
• Excel: .xlsx and .xls file support
• JSON: Structured data import

File Handling Options

• Watch Folders: Automatically process new files
• Batch Processing: Handle multiple files simultaneously
• File Validation: Verify data quality before processing
• Error Handling: Robust handling of malformed files

Supported Platforms

• Amazon S3: Direct connection to S3 buckets
• Google Cloud Storage: GCS bucket integration
• Azure Blob Storage: Azure storage account connections
• Dropbox/Google Drive: Personal cloud storage access