Building a Self-Service Internal Developer Platform (IDP) with Terraform and Backstage

Introduction: The Rise of Platform Engineering

In the fast-paced world of software delivery, the friction between Development (Dev) and Operations (Ops) is a persistent bottleneck. Developers need resources—databases, queues, test environments—instantly. Operations teams, prioritizing stability and governance, often act as gatekeepers, managing these requests via slow ticketing systems.

This misalignment drives the emergence of Platform Engineering, a discipline focused on building Internal Developer Platforms (IDPs) that enable developer self-service while enforcing organizational standards. HostingX IL is at the forefront of this shift in the Israeli market, helping companies architect IDPs that empower developers without sacrificing control.

According to Gartner, by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components, and tools for application delivery. The era of "you build it, you run it" is evolving into "we provide the platform, you build on it."

The Architecture of a Modern IDP

A robust IDP is not a single tool but a layered ecosystem designed to abstract complexity while maintaining governance:

HostingX IL advocates for an architecture where Backstage acts as the interface for Terraform. This combination allows developers to provision complex infrastructure by filling out a simple form, abstracting away the complexity of HCL code and cloud provider APIs.

Designing "Golden Paths" with Terraform Modules

The core concept of an IDP is the "Golden Path"—a pre-approved, supported way of building and deploying software. Instead of allowing developers to write arbitrary Terraform code (which carries security risks), Platform Engineering teams build reusable Terraform Modules that encapsulate best practices.

Example: The "Microservice Golden Path"

A microservice Golden Path might utilize a Terraform module that automatically provisions:

• Compute: An EKS deployment with Horizontal Pod Autoscaler or Lambda function with concurrency limits
• Networking: A private Application Load Balancer with specific Security Groups enforcing least-privilege access
• Data Layer: RDS database with automated backups, encryption at rest, and read replicas
• Observability: Datadog monitors, PagerDuty alerts, and CloudWatch Logs integration
• Security: IAM roles with least-privilege access, automatic secret rotation via AWS Secrets Manager

# Example: Microservice Golden Path Module
module "microservice" {
  source = "github.com/hostingx-il/terraform-modules//microservice"
  
  service_name    = "payment-api"
  team            = "payments"
  cost_center     = "engineering"
  environment     = "production"
  
  # Compute
  container_image = "ecr.io/payment-api:v1.2.3"
  cpu             = "1024"
  memory          = "2048"
  replicas        = 3
  
  # Database
  database_size   = "db.t3.medium"
  storage_gb      = 100
  multi_az        = true
  
  # Observability
  enable_datadog  = true
  sla_uptime      = "99.9"
  pagerduty_key   = var.pagerduty_payment_key
}

# Outputs automatically include:
# - Service URL
# - Database endpoint
# - IAM role ARN
# - CloudWatch Log Group

By utilizing these modules, HostingX IL ensures that every service deployed via the IDP is compliant by design. The developer does not need to know how to configure a VPC securely; they simply need to request a "High-Availability Microservice".

Integration Mechanics: Linking Backstage to Terraform

The technical integration between Backstage and Terraform relies on the "Scaffolder" pattern, which transforms user input into executable infrastructure code.

The Five-Step Integration Flow:

1. Template Definition: The platform team defines a YAML template in Backstage that specifies the input parameters (Service Name, Team, Cost Center, Environment).
2. User Interaction: Developer fills out a form in the Backstage UI with required parameters and optional configurations.
3. Trigger Event: When a developer clicks "Create," Backstage triggers a webhook to the CI/CD system (e.g., GitHub Actions).
4. Terraform Execution: The CI/CD pipeline clones a "template repository," hydrates it with the user's input variables, and runs terraform plan followed by terraform apply.
5. Feedback Loop: Once the infrastructure is provisioned, Terraform outputs the resource details (e.g., the database endpoint, service URL), which are sent back to the Backstage catalog and displayed to the developer.

HostingX IL specializes in building these custom integration flows, ensuring that the IDP feels like a seamless product rather than a disjointed set of tools. We handle the complexity of authentication, state management, and error handling so developers experience a "click and forget" provisioning experience.

Solving the "Day 2" Challenge

Many IDPs focus solely on creation ("Day 1"), leaving teams to struggle with updates and maintenance ("Day 2"). HostingX IL addresses this by integrating Terraform State Management into the IDP workflow.

Day 2 Operations in the IDP:

• Resource Updates: Developers can modify service parameters (e.g., increase database size) through the same Backstage form. The platform generates a new Terraform plan showing the diff and requires approval before applying.
• Drift Detection: Backstage can display a "Resource Health" scorecard by querying the Terraform state. If infrastructure has been modified manually in the console (violating IaC principles), the platform alerts the team and provides a one-click remediation.
• Decommissioning: When a service is no longer needed, the developer can "decommission" it through Backstage, which triggers terraform destroy in a controlled manner, ensuring all resources are cleaned up and costs stop accruing.
• Cost Visibility: Integration with Infracost shows the monthly cost of each service directly in the Backstage catalog, enabling teams to make informed decisions about resource allocation.

By using remote backends and separating state files per service (e.g., services/payment-api.tfstate, services/user-api.tfstate), the platform ensures that a developer updating Service A cannot accidentally impact Service B—a critical safety mechanism for multi-tenant platforms.

The Business Value of an IDP

Implementing an IDP with Terraform and Backstage delivers measurable ROI across multiple organizational dimensions:

Real-World Implementation: Case Study

Advanced Patterns: Progressive Delivery

HostingX IL implements progressive delivery patterns within the IDP framework, enabling advanced deployment strategies:

• Blue/Green Deployments: Terraform provisions parallel environments. Backstage orchestrates traffic cutover.
• Canary Releases: Integration with Flagger or Argo Rollouts for gradual traffic shifting with automatic rollback on SLO violations.
• Feature Flags: Backstage integrates with LaunchDarkly or Split.io to enable runtime feature toggles without redeployment.

HostingX IL: Your Platform Engineering Partner

Building an IDP requires a diverse skill set: React/TypeScript for the frontend, Go/Python for backend plugins, and deep Terraform expertise for the infrastructure engine. HostingX IL offers "Managed Platform" services that provide this expertise on demand.

Our Platform Engineering Service Includes:

• Golden Path Definition: We conduct workshops with your teams to identify the 5-10 service patterns that cover 80% of your use cases.
• Terraform Module Library: We build, test, and maintain production-grade modules with comprehensive documentation and examples.
• Backstage Customization: We deploy and customize Backstage, including custom plugins, themes, and integrations with your existing tools.
• CI/CD Integration: We build the orchestration layer that connects Backstage to your deployment pipelines with security scanning and approval gates.
• Training & Enablement: We train your platform team to maintain and extend the IDP, ensuring long-term sustainability.

Getting Started: The IDP Journey

HostingX IL recommends a phased approach to IDP implementation:

1. Phase 1 - Foundation (Weeks 1-4): Deploy Backstage, build first Golden Path (e.g., stateless microservice), pilot with 2-3 teams
2. Phase 2 - Expansion (Weeks 5-12): Add data layer paths (databases, queues), integrate observability, expand to 50% of engineering
3. Phase 3 - Maturity (Weeks 13-24): Advanced patterns (multi-region, DR), full organization adoption, self-service analytics

Conclusion: Empowering Developers, Protecting the Platform

The Internal Developer Platform represents the future of software delivery. By combining Backstage's exceptional developer experience with Terraform's infrastructure automation power, HostingX IL builds platforms that accelerate velocity while maintaining security, compliance, and cost efficiency.

For Israeli tech companies competing in global markets, the ability to provision production-grade infrastructure in minutes—not days—is a strategic differentiator. By partnering with HostingX IL, you can leapfrog the steep learning curve of Platform Engineering and deliver a world-class developer experience immediately.

The question is not whether your organization needs an IDP, but how quickly you can implement one. With HostingX IL, the answer is: faster than you think, and better than you imagined.