diff --git a/addons/custom-helm-charts.mdx b/addons/custom-helm-charts.mdx new file mode 100644 index 0000000..6ee897a --- /dev/null +++ b/addons/custom-helm-charts.mdx @@ -0,0 +1,166 @@ +--- +title: "Custom Helm Charts" +sidebarTitle: "Custom Helm Charts" +description: "Deploy external Helm charts for components not managed by Porter" +--- + +Sometimes you need to install external charts that are not managed by Porter. You can deploy custom Helm charts either through the Porter dashboard or directly via the Helm CLI. + +--- + +## Deploying via Porter + +Deploy Helm charts through the Porter dashboard for a managed experience with visibility into your deployments. + + + + Navigate to **Add-ons** in your Porter dashboard and select **Helm Chart**. + + + Enter the Helm repository URL and select the chart you want to deploy. + + + Choose the chart version you want to install. + + + Modify any values from the chart's default configuration as needed. + + + Click **Deploy** to install the chart to your cluster. + + + + +Since custom Helm charts install external components into your cluster, they fall outside of Porter's standard support. However, we'll do our best to help you troubleshoot issues. + + +### Managing deployed charts + +Once deployed, you can: +- View the chart status in the Add-ons tab +- Update values and redeploy +- Upgrade to newer chart versions +- Delete the chart when no longer needed + +--- + +## Deploying via Helm CLI + +For more control or CI/CD integration, you can deploy Helm charts directly using the Helm CLI. + +### Prerequisites + +1. Install the [Helm CLI](https://helm.sh/docs/intro/install/) +2. Configure kubectl to connect to your Porter cluster. Run: + ```bash + porter config set-cluster + ``` + And select the cluster from the dropdown. If there is only one + cluster in your project it will be automatically selected. + +### Deploying a chart + +```bash +# Add the Helm repository +helm repo add +helm repo update + +# Install the chart: +porter helm -- install / \ + --namespace \ + --values values.yaml +``` + +### Example: Installing NGINX Ingress + +```bash +# Add the ingress-nginx repository +porter helm -- repo add ingress-nginx https://kubernetes.github.io/ingress-nginx +porter helm -- repo update + +# Install the chart +porter helm -- install my-ingress ingress-nginx/ingress-nginx \ + --namespace ingress \ + --create-namespace +``` + +### Upgrading a release + +```bash +porter helm -- upgrade / \ + --namespace \ + --values values.yaml +``` + +### Listing releases + +```bash +porter helm -- list --all-namespaces +``` + +### Uninstalling a release + +```bash +porter helm -- uninstall --namespace +``` + +--- + +## Observability + +There is limited observability offered for third-party helm chart installations, consisting of logs available in the dashboard. + +For more complex scenarios, you can deploy a **Grafana** addon, which is already configured with the existing observability +stack deployed in the cluster. Grafana makes it possible to explore, query and build dashboard to monitor any charts or +workloads deployed in your cluster. + +--- + +## Best Practices + +### Use version pinning + +Always specify a chart version to ensure reproducible deployments: + +```bash +porter helm -- install my-release repo/chart --version 1.2.3 +``` + +### Store values in version control + +Keep your custom values files in version control alongside your application code. For example, for the +"custom-chart" helm chart, you can keep the following structure: + +`porter-custom-helm-chart-addons/custom-chart/chart.yaml` + +```yaml +# chart.yaml +chartUrl: https://custom-chart-repo.com +version: 1.0.0 +``` + +`porter-custom-helm-chart-addons/custom-chart/values.yaml` + +```yaml +# values.yaml +replicaCount: 3 +resources: + limits: + cpu: 100m + memory: 128Mi +``` +You can then use these values in CI to install the chart + +```bash +CHART_URL=$(yq e '.chartUrl' porter-custom-helm-chart-addons/custom-chart/chart.yaml) +VERSION=$(yq e '.version' porter-custom-helm-chart-addons/custom-chart/chart.yaml) + +porter helm -- install custom-chart $CHART_URL \ + --version $VERSION \ + --values porter-custom-helm-chart-addons/custom-chart/values.yaml +``` + +### Test in non-production first + +Before deploying to production, test custom charts in a development or staging cluster to verify compatibility with your Porter environment. + diff --git a/addons/datastores.mdx b/addons/datastores.mdx new file mode 100644 index 0000000..67dfc5d --- /dev/null +++ b/addons/datastores.mdx @@ -0,0 +1,198 @@ +--- +title: "Datastores" +sidebarTitle: "Datastores" +description: "Provision managed databases with automatic networking and security" +--- + +Porter simplifies database provisioning by automatically setting up all networking components between your cluster and Porter-provisioned databases. + + +Datastores are currently supported on **AWS only**. GCP and Azure datastore support is on the roadmap. + + +## AWS Architecture + +Datastores are provisioned in a VPC that is separate from the VPC of your clusters. Porter automatically: + +- Peers the datastore VPC to your cluster VPC +- Configures subnets, routing tables, and security groups +- Ensures traffic flows exclusively through private subnets + +This architecture keeps your database secure and accessible only from applications running in your cluster. + +--- + +## Setup + + + + Navigate to **Add-ons** in your Porter dashboard and select the datastore type you want to create (Postgres or Redis). + + + Configure your datastore settings including instance size, storage, and high availability options. + + + Porter creates an environment group with the connection details. Inject this environment group into your applications. + + + Deploy your application. It can now connect to the database using the injected environment variables. + + + +### Connecting from your laptop + +To connect to a datastore from your local machine, use the Porter CLI: + +```bash +porter datastore connect my-datastore +psql -h localhost -p -U -l +``` + + +If your cluster control plane access is set to private, using this command requires [Tailscale VPN](/cloud-accounts/tailscale) to be configured for your cluster. + + +--- + +## Postgres + +Postgres datastores can be deployed in different configurations depending on your needs: + +| Configuration | Use Case | Recommended For | +|--------------|----------|-----------------| +| **In-cluster** | Quick setup for development | Dev/staging environments | +| **Single RDS instance (Multi-AZ)** | Standard managed database | Production workloads | +| **Aurora cluster (Multi-AZ)** | Auto-scaling storage and enhanced failover | Production workloads with stringent high-availability requirements | + +### In-cluster Postgres + +Deploys Postgres as a container within your cluster. This is the fastest way to get started but is **not recommended for production data**. + +### RDS Instance + +Provisions a standard Amazon RDS instance with Multi-AZ deployment for automatic failover. This is the recommended option for most production workloads. + +### Aurora Cluster + +Aurora provides: +- Automatic storage autoscaling +- Enhanced failover capabilities +- High availability settings + +You can create an Aurora datastore with a single instance or with an additional read replica. + +#### Read Replicas + +To enable a read replica, select the **HA toggle** when creating the datastore. + +With read replicas: +- The dashboard displays connection details for both primary and replica +- Modifications automatically failover the primary and promote the replica +- This ensures minimum downtime during operations + +### Configuration + +The following table outlines the configurable fields and behaviors for each datastore type during creation and updates: + +| Configuration | In-cluster | Standard RDS | Aurora | +| :--- | :--- | :--- | :--- | +| **Connected cluster** | Local (via K8s Service) | External (via VPC Peering) | External (via VPC Peering) | +| **Region** | Matches connected cluster | Matches connected cluster | Matches connected cluster | +| **Database name** | - | User-defined | User-defined | +| **Master username** | - | User-defined | User-defined | +| **Postgres version** | - | Postgres 12-18 | Postgres 12-18 | +| **Instance type** | CPU/RAM Limits | All RDS compatible instances | All Aurora compatible classes, excepts serverless | +| **Allocated storage** | **Fixed** (Cannot be modified) | **Modifiable** (Increase only) | **Managed** (Auto-scales) | +| **Snapshot restore** | - | From RDS snapshot | From Aurora snapshot | +| **Cloning** | - | - | Fast Database Cloning | +--- + +## Redis + +Redis datastores can be provisioned in different configurations: + +| Configuration | Use Case | Recommended For | +|--------------|----------|-----------------| +| **In-cluster** | Quick setup for development | Dev/staging environments | +| **Elasticache replication group** | Managed cache with automatic failover | Production workloads | + +### In-cluster Redis + +Deploys Redis as a container within your cluster. This is the fastest way to get started but is **not recommended for production data**. + +### Elasticache Replication Group + +Provisions an Amazon Elasticache replication group with: +- Primary and reader replica by default +- Automatic failover if the primary fails +- Minimal downtime during modifications + +--- + +## Monitoring + +You can monitor the performance of your database from the Porter dashboard. Metrics are available in the "Metrics" tab when opening a datastore. +The metrics that are currently displayed are: + +- CPU utilization +- RAM utilization +- Storage capacity + +--- + +## Disaster Recovery + +Porter supports some options for disaster recovery of RDS datastores. + +### Restoring from a snapshot + +You can restore a snapshot to a new datastore, and the datastore will be accessible from the applications running in your cluster. This can significantly +reduce the time to recovery during an emergency. + + + + Create a new datastore in the dashboard. In the creation form, click on **"Enable snapshot restore"** + + + Select one of the available snapshots to restore or enter the id manually. Only snapshots in the same region as the database are listed. + + + Create the datastore. This will start the process of restoring the snapshot. + + + +### Cloning an Aurora cluster + +Aurora clusters support cloning an existing cluster using fast-cloning. This process is faster than restoring from a snapshot, and can be used +to recover from an emergency, or to quickly create copies of your database for experiments. + + + + Create a new datastore in the dashboard. In the creation form, click on **"Enable database cloning"** + + + Select one of the existing Aurora clusters or enter the identifier manually. Only clusters in the same region as the selected one are listed. + + + Create the datastore. This will start the process of cloning the cluster. + + + +--- + +## Compliance + +If the compliance feature is enabled for your project, Porter automatically configures monitoring alarms for RDS and Aurora datastores: + +- CPU utilization alarms +- Memory utilization alarms +- Storage capacity alarms + +These alarms help ensure your databases remain healthy and within operational thresholds. + +## Roadmap + +The following features are not yet supported natively in Porter, but reach out to the support team for help setting them up. + +- Connection pooling +- External access to datastores diff --git a/addons/overview.mdx b/addons/overview.mdx new file mode 100644 index 0000000..51ed7a0 --- /dev/null +++ b/addons/overview.mdx @@ -0,0 +1,86 @@ +--- +title: "Add-ons" +sidebarTitle: "Overview" +description: "Extend your Porter cluster with datastores, monitoring, and more" +--- + +Add-ons extend your Porter cluster with additional infrastructure components like databases, monitoring tools, persistent storage, and custom Helm charts. + +## Available Add-ons + +### Databases + +| Add-on | Description | +|--------|-------------| +| **Postgres** | An object-relational database system | +| **Redis** | An in-memory key-value database | + +[Learn more about Datastores →](/addons/datastores) + +### Monitoring + +| Add-on | Description | +|--------|-------------| +| **Datadog** | Pipe logs, metrics, and APM data from your workloads | +| **New Relic** | Monitor your applications and infrastructure | +| **Grafana** | An open source analytics and monitoring tool | +| **Langfuse** | An open source LLM engineering platform | +| **Helicone AI Gateway** | An open source AI gateway for LLM requests | + +[Learn more about Third-party Observability →](/addons/third-party-observability) + +### Logging + +| Add-on | Description | +|--------|-------------| +| **Mezmo** | A popular logging management system | + +### Analytics + +| Add-on | Description | +|--------|-------------| +| **Metabase** | An open source business intelligence tool | +| **Quivr** | Your second brain, empowered by generative AI | +| **n8n** | An open source workflow engine | + +### Storage + +| Add-on | Description | +|--------|-------------| +| **Persistent Disk** | A persistent disk that can be attached to apps for data storage | + +[Learn more about Storage →](/addons/storage) + +### Custom + +| Add-on | Description | +|--------|-------------| +| **Helm Chart** | Install any Helm chart from a public repository | + +[Learn more about Custom Helm Charts →](/addons/custom-helm-charts) + +--- + +## Cloud Provider Support + +| Add-on | AWS | GCP | Azure | +|--------|-----|-----|-------| +| **Datastores (Postgres, Redis)** | ✅ | Coming soon | Coming soon | +| **Monitoring & Analytics** | ✅ | ✅ | ✅ | +| **Custom Helm Charts** | ✅ | ✅ | ✅ | +| **Storage (Persistent Disk)** | ✅ | Coming soon | Coming soon | + +--- + +## Managing Add-ons + +Add-ons are managed through the **Add-ons** tab in your Porter dashboard. From there you can: + +- Create new add-ons +- View and modify existing add-on configurations +- Connect add-ons to your applications via environment groups +- Delete add-ons when no longer needed + + +Deleting an add-on may cause connected applications to lose access to the resource. Ensure you've migrated any data before deleting datastores or storage add-ons. + diff --git a/addons/storage.mdx b/addons/storage.mdx new file mode 100644 index 0000000..e3db3f8 --- /dev/null +++ b/addons/storage.mdx @@ -0,0 +1,97 @@ +--- +title: "Storage" +sidebarTitle: "Storage" +description: "Add persistent storage with shared disks for your applications" +--- + +Porter makes it easy to provision persistent storage that can be shared across multiple services. This is useful for read-intensive applications and workloads with low-latency read requirements. + + +Persistent storage is currently supported on **AWS only** (EFS). GCP and Azure storage support is on the roadmap. + + +--- + +## Persistent Disk (EFS) + +Amazon Elastic File System (EFS) provides a shared filesystem that multiple services can mount simultaneously. Any services mounting the disk can read and write to the same files. + +### Use Cases + +- **Shared file storage**: Multiple services need access to the same files +- **Read-intensive workloads**: Cache frequently accessed data on disk +- **Media processing**: Store uploaded files for processing by multiple workers +- **Machine learning**: Share model files across inference services + +### Setup + + + + Navigate to **Add-ons** in your Porter dashboard and create a **Persistent Disk** add-on. + + + Go to your application's **Services** tab, select the service, and navigate to **Advanced Settings**. + + Enable **Persistent disk**. + + + Enter the name of the persistent disk add-on you created. + + + Deploy your application. The disk will be mounted automatically. + + + +### Accessing the Disk + +Once configured, your service can access the shared disk at: + +``` +/data/efs/ +``` + +All services with the persistent disk enabled will have read and write access to this directory. + +### Example + +If you have two services (`api` and `worker`) both mounting the same persistent disk 'my-disk': + +- `api` writes a file to `/data/api/my-disk/uploads/image.png` +- `worker` can read the same file from `/data/worker/my-disk/image.png` + +--- + +## Best Practices + +### Use for appropriate workloads + +EFS is optimized for throughput rather than IOPS. It's best suited for: +- Large file reads and writes +- Shared access patterns +- Workloads that can tolerate slightly higher latency than local disk + +For high-IOPS workloads like databases, use [Datastores](/addons/datastores) instead. + +### Monitor storage usage + +Keep track of storage usage to avoid unexpected costs. EFS charges based on the amount of data stored. + +### Plan for data migration + +If you need to move data off EFS, plan your migration strategy before deleting the add-on. + +--- + +## Deleting a Persistent Disk + + +Deleting the persistent disk add-on will remove all mounts to the disk. Any applications will lose access to the files stored on the disk. + + +Before deleting: + +1. Migrate any important data to another storage location +2. Update your services to remove the persistent disk configuration +3. Redeploy affected services +4. Delete the persistent disk add-on + diff --git a/addons/third-party-observability.mdx b/addons/third-party-observability.mdx new file mode 100644 index 0000000..30672f9 --- /dev/null +++ b/addons/third-party-observability.mdx @@ -0,0 +1,53 @@ +--- +title: "Third-party Observability" +sidebarTitle: "Third-party Observability" +description: "Integrate external monitoring and observability platforms" +--- + +Porter integrates with popular observability platforms to provide application-level monitoring, logging, and alerting beyond the built-in cluster observability features. + +## Supported Platforms + + + + Full-stack monitoring with APM, logs, and infrastructure metrics + + + Application performance monitoring and alerting + + + Dashboards and visualization for metrics and logs + + + Open source LLM engineering platform for tracing and analytics + + + Open source AI gateway for LLM request monitoring + + + + +## What You Get + +Integrating a third-party observability platform provides: + +- **Application Performance Monitoring (APM)**: Trace requests across services +- **Log aggregation**: Centralized logging with search and filtering +- **Custom metrics**: Track business and application-specific metrics +- **Alerting**: Get notified when metrics exceed thresholds +- **Dashboards**: Visualize system health and performance trends + +## Comparison with Built-in Observability + +| Feature | Porter Built-in | Third-party Platforms | +|---------|----------------|----------------------| +| Pod status and metrics | ✅ | ✅ | +| Node metrics | ✅ | ✅ | +| Application logs | Basic | Advanced search and filtering | +| Distributed tracing | ❌ | ✅ | +| Custom dashboards | ❌ | ✅ | +| Alerting | ❌ | ✅ | +| Long-term retention | Limited | Configurable | + +For production workloads, we recommend integrating at least one third-party observability platform to get comprehensive visibility into your applications. + diff --git a/mint.json b/mint.json index 3f20507..d7b68c9 100644 --- a/mint.json +++ b/mint.json @@ -116,6 +116,16 @@ } ] }, + { + "group": "Add-ons", + "pages": [ + "addons/overview", + "addons/datastores", + "addons/custom-helm-charts", + "addons/third-party-observability", + "addons/storage" + ] + }, { "group": "Command Line Interface (CLI)", "pages": [