))
Welcome to The Data Flowcast: Mastering Airflow for Data Engineering & AI — the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward. Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workflow management system to meet the ever-evolving needs of data engineering and AI ecosystems. Podcast Webpage: https://www.astronomer.io/podcast/
…
continue reading
1
AI-Powered Vehicle Automation at Ford Motor Company with Serjesh Sharma
26:11
26:11
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
26:11
Harnessing data at scale is the key to driving innovation in autonomous vehicle technology. In this episode, we uncover how advanced orchestration tools are transforming machine learning operations in the automotive industry. Serjesh Sharma, Supervisor ADAS Machine Learning Operations (MLOps) at Ford Motor Company, joins us to discuss the challenge…
…
continue reading
1
From Task Failures to Operational Excellence at GumGum with Brendan Frick
24:06
24:06
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
24:06
Data failures are inevitable but how you manage them can define the success of your operations. In this episode, we dive deep into the challenges of data engineering and AI with Brendan Frick, Senior Engineering Manager, Data at GumGum. Brendan shares his unique approach to managing task failures and DAG issues in a high-stakes ad-tech environment.…
…
continue reading
1
From Sensors to Datasets: Enhancing Airflow at Astronomer with Maggie Stark and Marion Azoulai
22:25
22:25
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
22:25
A 13% reduction in failure rates — this is how two data scientists at Astronomer revolutionized their data pipelines using Apache Airflow.In this episode, we enter the world of data orchestration and AI with Maggie Stark and Marion Azoulai, both Senior Data Scientists at Astronomer. Maggie and Marion discuss how their team re-architected their use …
…
continue reading
1
Mastering Data Orchestration with Airflow at M Science with Ben Tallman
24:36
24:36
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
24:36
Mastering the flow of data is essential for driving innovation and efficiency in today’s competitive landscape. In this episode, we explore the evolution of data orchestration and the pivotal role of Apache Airflow in modern data workflows.Ben Tallman, Chief Technology Officer at M Science, joins us and shares his extensive experience with Airflow,…
…
continue reading
Welcome to The Data Flowcast: Mastering Airflow for Data Engineering & AI — the podcast where we keep you up to date with insights and ideas propelling the Airflow community forward.Join us each week, as we explore the current state, future and potential of Airflow with leading thinkers in the community, and discover how best to leverage this workf…
…
continue reading
1
Enhancing Business Metrics With Airflow at Artlist with Hannan Kravitz
23:51
23:51
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:51
Data orchestration is revolutionizing the way companies manage and process data. In this episode, we explore the critical role of data orchestration in modern data workflows and how Apache Airflow is used to enhance data processing and AI model deployment.Hannan Kravitz, Data Engineering Team Leader at Artlist, joins us to share his insights on lev…
…
continue reading
1
Cutting-Edge Data Engineering at Teya with Alexandre Magno Lima Martins
23:46
23:46
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:46
Data engineering is constantly evolving and staying ahead means mastering tools like Apache Airflow. In this episode, we explore the world of data engineering with Alexandre Magno Lima Martins, Senior Data Engineer at Teya. Alexandre talks about optimizing data workflows and the smart solutions they've created at Teya to make data processing easier…
…
continue reading
1
Airflow Strategies for Business Efficiency at Campbell with Larry Komenda
26:10
26:10
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
26:10
Managing data workflows well can change the game for any company. In this episode, we talk about how Airflow makes this possible. Larry Komenda, Chief Technology Officer at Campbell, shares how Airflow supports their operations and improves efficiency.Larry discusses his role at Campbell, their switch to Airflow, and its impact. We look at their st…
…
continue reading
1
How Laurel Uses Airflow To Enhance Machine Learning Pipelines with Vincent La and Jim Howard
23:58
23:58
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:58
The world of timekeeping for knowledge workers is transforming through the use of AI and machine learning. Understanding how to leverage these technologies is crucial for improving efficiency and productivity.In this episode, we’re joined by Vincent La, Principal Data Scientist at Laurel, and Jim Howard, Principal Machine Learning Engineer at Laure…
…
continue reading
1
How Vibrant Planet's Self-Healing Pipelines Revolutionize Data Processing
23:51
23:51
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:51
Discover the cutting-edge methods Vibrant Planet uses to revolutionize geospatial data processing and resource management.In this episode, we delve into the intricacies of scaling geospatial data processing and resource allocation with experts from Vibrant Planet. Joining us are Cyrus Dukart, Engineering Lead, and David Sacerdote, Staff Software En…
…
continue reading
1
The Future of AI in Data Engineering With Astronomer’s Julian LaNeve and David Xue
23:36
23:36
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:36
The world of data orchestration and machine learning is rapidly evolving, and tools like Apache Airflow are at the forefront of these changes. Understanding how to effectively utilize these tools can significantly enhance data processing and AI model deployment.This episode features Julian LaNeve, CTO at Astronomer, and David Xue, Machine Learning …
…
continue reading
1
The Power of Airflow in Modern Data Environments at Wynn Las Vegas with Siva Krishna Yetukuri
24:31
24:31
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
24:31
Understanding the critical role of data integration and management is essential for driving business success, particularly in a dynamic environment like a luxury casino resort.In this episode, we sit down with Siva Krishna Yetukuri, Cloud Data Architect at Wynn Las Vegas, to explore how Airflow and other tools are transforming data workflows and cu…
…
continue reading
1
Powering the Texas Rangers World Series Win With AI on Airflow with Alexander Booth
23:38
23:38
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
23:38
The integration of data and AI in sports is transforming how teams strategize and perform. Understanding how to harness this technology is key to staying competitive in the rapidly evolving landscape of baseball.In this episode, we sit down with Alexander Booth, Assistant Director of Research and Development at Texas Rangers Baseball Club, to explo…
…
continue reading
1
Expanding the Data Engineering Toolkit at Reddit
45:48
45:48
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
45:48
Welcome back to the Airflow Podcast.This week, we met up with Ben Wisegarver, a staff data scientist at Reddit who runs their data warehousing and data engineering functions.Reddit users generate petabytes of data every day that needs to be processed, stored, and analyzed by a wide breadth of backend services. Our conversation with Ben touches on e…
…
continue reading
1
GDPR, Self-Service Data, and Infrastructure Automation with Typeform
31:26
31:26
बाद में चलाएं
बाद में चलाएं
सूचियाँ
पसंद
पसंद
31:26
Welcome back to the Airflow Podcast.This week, we met up with Albert Franzi and Carlos Escura from Typeform. Typeform is a tool that allows you to build beautiful interactive forms that you can use for a wide variety of use cases, including customer surveys, employee engagement, product feedback, and market research to name a few. In our conversati…
…
continue reading
After a bit of a break, we're back with the third official episode bundle of The Airflow Podcast. In this batch, we'll get a little bit deeper with current Airflow users and maintainers on core fundamental concepts in data engineering, architectures for operating modern data platforms at scale, and the process of maintaining and operating Airflow, …
…
continue reading
This week, we linked up with Airflow release manager, core committer, and Astronomer platform engineer Ash Berlin-Taylor to discuss the Airflow 2.0 roadmap [1]. There is some great stuff in the works around performance, autoscaling, and usability that we're excited about. In this episode, Ash lends his thoughts on the design, implementation, and va…
…
continue reading
This week, we had the pleasure of meeting up with Jarek Potiuk, Principal Software Engineer at Polidea and Apache Airflow committer, to discuss his most recent contribution to the community, Airflow Breeze. Jarek deeply values developer productivity and realized while building a team of Airflow committers that, in order to open a PR on the project,…
…
continue reading
This episode kicks off season 2 of The Airflow Podcast. In this next season, we'll focus on the future of Airflow and chat with leading members of the community to paint a picture of what's to come. We're pumped to be diving back into this project and look forward to the great conversations we have lined up.This week, we chatted with James Malone, …
…
continue reading
This week, we met up with Ash Berlin-Taylor to discuss the recent 1.10 release, what it's like to be a release manager for an open source project, Airflow's bid to graduate from incubating status, and the next phase of Airflow project development.As mentioned in our podcast intro, we at Astronomer are hiring Data Engineers who are passionate about …
…
continue reading
This time, we met up with WePay's Joy Gao to talk through her work on the RBAC components in the recent Airflow 1.10 release. We dove deep into what inspired her work and took some time to discuss what it's like to be a woman contributing to a predominately male open-source community. Hope you enjoy!If you'd like to get started using Airflow in you…
…
continue reading
In this episode, we dove into the relationship between Airflow and Kuberenetes and interviewed Daniel Imberman, Senior Software Engineer at Bloomberg (1:30), and Greg Neiheisel, CTO here at Astronomer (37:31). Daniel has done most of the work on the Kubernetes executor for Airflow and Greg plans to take on a chunk of the development going forward, …
…
continue reading
This week, we’ll examine conversations with both old guests and new to paint a comprehensive picture of Airflow’s pain points. While we still undoubtedly believe that Airflow is the future of ETL, it’s important to acknowledge that any incubating project will have issues, and bringing those issues to the forefront of the community’s attention will …
…
continue reading
On this episode, we linked up with Erik Bernhardsson (@erikbern), creator of Luigi and CTO of Better Mortgage. We chatted about everything from the motivations behind Luigi's creation and his current thoughts on Airflow- we hope you enjoy!Check out:- Erik's blog at erikbern.com- Our open-source library of Airflow plugins at github.com/airflow-plugi…
…
continue reading
In this episode, we dive into Airflow Best Practices and include longer portions of interviews with Alan Cruickshank (1:30), Business Insights and Data Manager at Tails.com, Chris Riccomini (7:27), Principal Software Engineer at WePay, and Bolke de Bruin(31:45), Head of Advanced Analytics Technology at ING. Hope you enjoy!We're still working to get…
…
continue reading
