We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. Key Responsibilities - Design and architect Databricks Lakehouse solutions for large-scale enterprise data platforms. - Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization. - Define Databricks workspace architecture, cluster strategies, security models, and operational standards. - Design and develop scalable data pipelines using Databricks, Apache Spark, PySpark, SQL, and Delta Lake . - Implement data ingestion frameworks supporting batch and real-time data processing. - Design Delta Lake architectures including: - Delta tables - Medallion architecture (Bronze/Silver/Gold layers) - Data quality frameworks - Schema evolution and optimization - Perform Databricks performance tuning: - Cluster optimization - Spark job optimization - Query tuning - Cost optimization - Implement Databricks governance and security using: - Unity Catalog - Access controls - Data lineage - Audit capabilities - Lead migration projects from legacy data platforms to Databricks Lakehouse. - Define best practices for Databricks CI/CD, DevOps, and environment management. - Collaborate with data engineers, data scientists, analysts, and enterprise architects. - Provide technical guidance, architecture documentation, and design reviews. Skills & Experience - Strong experience in data engineering, big data, or solution architecture. - Strong hands-on experience with Databricks Platform . - Expert knowledge of: - Apache Spark - PySpark / Scala - Delta Lake - SQL - Data Lakehouse architecture - ETL/ELT development - Experience designing and delivering enterprise Databricks solutions. - Strong experience with cloud platforms: - Azure Databricks (preferred) - AWS Databricks - Google Cloud Databricks - Experience with: - Databricks Workflows - Jobs orchestration - Notebooks - Repos - Cluster management - Unity Catalog Preferred Skills - Databricks Certified Data Engineer Professional / Databricks Certified Architect. - Experience with: - MLflow - Databricks Machine Learning - Feature Store - Model deployment - MLOps - Knowledge of streaming technologies: - Structured Streaming - Kafka - Experience with data governance, compliance, and enterprise security frameworks. Key Competencies - Strong Databricks architecture and implementation expertise. - Ability to design scalable and cost-efficient Lakehouse solutions. - Strong troubleshooting and performance optimization skills. - Ability to lead technical discussions with enterprise stakeholders.

Senior Databricks Solution Architect

similar openings