{"id":4281,"date":"2022-08-05T11:15:16","date_gmt":"2022-08-05T11:15:16","guid":{"rendered":"https:\/\/otus.ru\/journal\/?p=4281"},"modified":"2022-08-05T11:15:17","modified_gmt":"2022-08-05T11:15:17","slug":"dataops-engineering-chitaem-na-habr","status":"publish","type":"post","link":"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/","title":{"rendered":"DataOps Engineering: \u0447\u0438\u0442\u0430\u0435\u043c \u043d\u0430 \u0425\u0430\u0431\u0440"},"content":{"rendered":"<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_61 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title \" >\u0421\u043e\u0434\u0435\u0440\u0436\u0430\u043d\u0438\u0435<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/#%D0%9F%D1%80%D0%BE%D0%B5%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5_Data_Pipelines_%D0%B2_Apache_Airflow\" title=\"\u041f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 Data Pipelines \u0432 Apache Airflow\">\u041f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 Data Pipelines \u0432 Apache Airflow<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/#%D0%9A%D0%B0%D0%BA_%D0%BE%D0%B1%D1%80%D0%B0%D0%B1%D0%B0%D1%82%D1%8B%D0%B2%D0%B0%D1%82%D1%8C_%D0%BE%D0%B1%D1%8A%D0%B5%D0%BA%D1%82%D1%8B_Avro_%D1%81_%D0%BF%D0%BE%D0%BC%D0%BE%D1%89%D1%8C%D1%8E_%D0%B4%D0%B0%D1%82%D0%B0%D1%81%D0%B5%D1%82%D0%BE%D0%B2_Spark_32_Scala_212\" title=\"\u041a\u0430\u043a \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0431\u044a\u0435\u043a\u0442\u044b Avro \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u043e\u0432 Spark 3.2 &amp; Scala 2.12\">\u041a\u0430\u043a \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0431\u044a\u0435\u043a\u0442\u044b Avro \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u043e\u0432 Spark 3.2 &amp; Scala 2.12<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/#DevOps_Data_%D0%A1%D0%B8%D0%BD%D0%B5%D1%80%D0%B3%D0%B8%D1%8F_%D0%B4%D0%B2%D1%83%D1%85_%D0%BC%D0%B8%D1%80%D0%BE%D0%B2_DataOps\" title=\"DevOps + Data: \u0421\u0438\u043d\u0435\u0440\u0433\u0438\u044f \u0434\u0432\u0443\u0445 \u043c\u0438\u0440\u043e\u0432 = DataOps\">DevOps + Data: \u0421\u0438\u043d\u0435\u0440\u0433\u0438\u044f \u0434\u0432\u0443\u0445 \u043c\u0438\u0440\u043e\u0432 = DataOps<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/#%D0%9F%D0%BB%D0%B0%D1%82%D1%84%D0%BE%D1%80%D0%BC%D0%B0_%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D1%85_%D0%BA%D0%B0%D0%BA_%D1%83%D1%81%D0%BB%D1%83%D0%B3%D0%B0\" title=\"\u041f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 \u043a\u0430\u043a \u0443\u0441\u043b\u0443\u0433\u0430\">\u041f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 \u043a\u0430\u043a \u0443\u0441\u043b\u0443\u0433\u0430<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/otus.ru\/journal\/dataops-engineering-chitaem-na-habr\/#%D0%A1%D0%BC%D0%BE%D1%82%D1%80%D0%B8%D1%82%D0%B5_%D0%B7%D0%B0%D0%BF%D0%B8%D1%81%D0%B8_%D0%BE%D1%82%D0%BA%D1%80%D1%8B%D1%82%D1%8B%D1%85_%D1%83%D1%80%D0%BE%D0%BA%D0%BE%D0%B2_%D0%BA%D1%83%D1%80%D1%81%D0%B0_%C2%ABDataOps_Engineer%C2%BB\" title=\"\u0421\u043c\u043e\u0442\u0440\u0438\u0442\u0435 \u0437\u0430\u043f\u0438\u0441\u0438 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0443\u0440\u043e\u043a\u043e\u0432 \u043a\u0443\u0440\u0441\u0430 &#171;DataOps Engineer&#187;\">\u0421\u043c\u043e\u0442\u0440\u0438\u0442\u0435 \u0437\u0430\u043f\u0438\u0441\u0438 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0443\u0440\u043e\u043a\u043e\u0432 \u043a\u0443\u0440\u0441\u0430 &#171;DataOps Engineer&#187;<\/a><\/li><\/ul><\/nav><\/div>\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%D0%9F%D1%80%D0%BE%D0%B5%D0%BA%D1%82%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5_Data_Pipelines_%D0%B2_Apache_Airflow\"><\/span><a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/679104\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u041f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 Data Pipelines \u0432 Apache Airflow<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>\u042d\u0442\u0430 \u0441\u0442\u0430\u0442\u044c\u044f \u043f\u043e\u0441\u0432\u044f\u0449\u0435\u043d\u0430 \u043a\u0440\u0430\u0442\u043a\u043e\u043c\u0443 \u0432\u0432\u0435\u0434\u0435\u043d\u0438\u044e \u0432 Airflow \u0438 \u0448\u0430\u0433\u043e\u0432 \u043f\u043e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044e \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0435 \u043a\u043e\u043d\u0432\u0435\u0439\u0435\u0440\u043e\u0432 \u0434\u0430\u043d\u043d\u044b\u0445 (Data Pipelines). \u0421\u043d\u0430\u0447\u0430\u043b\u0430 \u043c\u044b \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u043c \u0438 \u0441\u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u0443\u0435\u043c Airflow. \u0417\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430 DAG \u0432 Airflow. \u041d\u0430\u0448\u0435\u0439 \u0446\u0435\u043b\u044c\u044e \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u043f\u043e\u043d\u0438\u043c\u0430\u043d\u0438\u0435 \u0440\u0430\u0437\u0432\u0435\u0440\u0442\u044b\u0432\u0430\u043d\u0438\u044f Airflow \u0438 \u0431\u0430\u0437\u043e\u0432\u043e\u0439 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 DAG.<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%D0%9A%D0%B0%D0%BA_%D0%BE%D0%B1%D1%80%D0%B0%D0%B1%D0%B0%D1%82%D1%8B%D0%B2%D0%B0%D1%82%D1%8C_%D0%BE%D0%B1%D1%8A%D0%B5%D0%BA%D1%82%D1%8B_Avro_%D1%81_%D0%BF%D0%BE%D0%BC%D0%BE%D1%89%D1%8C%D1%8E_%D0%B4%D0%B0%D1%82%D0%B0%D1%81%D0%B5%D1%82%D0%BE%D0%B2_Spark_32_Scala_212\"><\/span><a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/650125\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u041a\u0430\u043a \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0431\u044a\u0435\u043a\u0442\u044b Avro \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u043e\u0432 Spark 3.2 &amp; Scala 2.12<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>\u0420\u0430\u0437\u0431\u0435\u0440\u0435\u043c, \u043a\u0430\u043a \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0431\u044a\u0435\u043a\u0442\u044b JVM, \u0441\u0433\u0435\u043d\u0435\u0440\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u044b\u0435 \u0438\u0437 \u0441\u0445\u0435\u043c Avro, \u0432 \u0434\u0430\u0442\u0430\u0441\u0435\u0442\u0430\u0445 Spark. \u0412\u043c\u0435\u0441\u0442\u0435 \u0441 \u044d\u0442\u0438\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c, \u043a\u0430\u043a \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u0442\u044c \u043a\u043e\u0434 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0448\u0430\u0431\u043b\u043e\u043d\u0430 \u0444\u0443\u043d\u043a\u0446\u0438\u043e\u043d\u0430\u043b\u044c\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f &#171;\u043a\u043b\u0430\u0441\u0441 \u0442\u0438\u043f\u043e\u0432&#187; (type class) \u043d\u0430 \u044f\u0437\u044b\u043a\u0435 Scala.<\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"DevOps_Data_%D0%A1%D0%B8%D0%BD%D0%B5%D1%80%D0%B3%D0%B8%D1%8F_%D0%B4%D0%B2%D1%83%D1%85_%D0%BC%D0%B8%D1%80%D0%BE%D0%B2_DataOps\"><\/span><a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/648683\/\" target=\"_blank\" rel=\"noreferrer noopener\">DevOps + Data: \u0421\u0438\u043d\u0435\u0440\u0433\u0438\u044f \u0434\u0432\u0443\u0445 \u043c\u0438\u0440\u043e\u0432 = DataOps<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>\u0412 \u044d\u0442\u043e\u0439 \u0437\u0430\u043c\u0435\u0442\u043a\u0435 \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c:<\/p>\n\n\n\n<ol><li>\u041f\u043e\u0447\u0435\u043c\u0443 \u043f\u043e\u044f\u0432\u0438\u043b\u0430\u0441\u044c \u043f\u043e\u0442\u0440\u0435\u0431\u043d\u043e\u0441\u0442\u044c \u0432 DataOps;<\/li><li>\u0422\u0440\u0438 \u043f\u0440\u043e\u0441\u0442\u044b\u0445 \u0438\u0434\u0435\u0438, \u043d\u0430 \u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u0444\u043e\u043a\u0443\u0441\u0438\u0440\u0443\u0435\u0442\u0441\u044f DataOps:<ul><li>\u0420\u0430\u0431\u043e\u0442\u0430 \u0441 \u0434\u0430\u043d\u043d\u044b\u043c\u0438 \u0434\u043e\u043b\u0436\u043d\u0430 \u0431\u044b\u0442\u044c\u00a0\u0432\u043e\u0441\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u043c\u043e\u0439;<\/li><li>\u0410\u043d\u0430\u043b\u0438\u0442\u0438\u043a\u0430 \u043a\u0430\u043a \u043a\u043e\u0434;<br>(\u041a\u0430\u0436\u0435\u0442\u0441\u044f, \u0447\u0442\u043e \u044d\u0442\u043e \u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0438 \u0432\u044b\u0442\u0435\u043a\u0430\u0435\u0442 \u0438\u0437 \u043f.1., \u043d\u043e \u043d\u0435 \u0432\u0441\u0435 \u0442\u0430\u043a \u043f\u0440\u043e\u0441\u0442\u043e);<\/li><li>\u0414\u0443\u043c\u0430\u0439\u0442\u0435 \u043e \u0434\u0430\u043d\u043d\u044b\u0445 \u043a\u0430\u043a \u043e\u00a0\u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0435.<\/li><\/ul><\/li><\/ol>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%D0%9F%D0%BB%D0%B0%D1%82%D1%84%D0%BE%D1%80%D0%BC%D0%B0_%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D1%85_%D0%BA%D0%B0%D0%BA_%D1%83%D1%81%D0%BB%D1%83%D0%B3%D0%B0\"><\/span><a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/648435\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u041f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0430 \u0434\u0430\u043d\u043d\u044b\u0445 \u043a\u0430\u043a \u0443\u0441\u043b\u0443\u0433\u0430<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p>\u041a\u043e\u0433\u0434\u0430 \u043c\u044b \u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u0443\u0435\u043c \u0438 \u0441\u043e\u0437\u0434\u0430\u0435\u043c \u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u0443 \u0434\u0430\u043d\u043d\u044b\u0445, \u0442\u043e \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u043c \u043d\u0430\u0434 \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0435\u043d\u0438\u0435\u043c \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e\u0441\u0442\u0435\u0439 \u0438 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u044b \u0434\u0440\u0443\u0433\u0438\u043c \u043a\u043e\u043c\u0430\u043d\u0434\u0430\u043c \u0434\u043b\u044f \u0440\u0430\u0437\u0432\u0438\u0442\u0438\u044f \u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u043e\u0432.\u00a0<a href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/648435\/\" target=\"_blank\" rel=\"noreferrer noopener\">\u0427\u0438\u0442\u0430\u0442\u044c \u0434\u0430\u043b\u0435\u0435.<\/a><\/p>\n\n\n\n<hr class=\"wp-block-separator\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"%D0%A1%D0%BC%D0%BE%D1%82%D1%80%D0%B8%D1%82%D0%B5_%D0%B7%D0%B0%D0%BF%D0%B8%D1%81%D0%B8_%D0%BE%D1%82%D0%BA%D1%80%D1%8B%D1%82%D1%8B%D1%85_%D1%83%D1%80%D0%BE%D0%BA%D0%BE%D0%B2_%D0%BA%D1%83%D1%80%D1%81%D0%B0_%C2%ABDataOps_Engineer%C2%BB\"><\/span>\u0421\u043c\u043e\u0442\u0440\u0438\u0442\u0435 \u0437\u0430\u043f\u0438\u0441\u0438 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0443\u0440\u043e\u043a\u043e\u0432 \u043a\u0443\u0440\u0441\u0430 <a href=\"https:\/\/otus.ru\/lessons\/dataops\/https:\/\/otus.ru\/lessons\/ml-bigdata\/?utm_source=oj&amp;utm_medium=affilate&amp;utm_campaign=dataops&amp;mxm=[[hash_metrika]]&amp;relogin=True&amp;token=[[token]]\" target=\"_blank\" rel=\"noreferrer noopener\">&#171;DataOps Engineer&#187;<\/a><span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-layout-1 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"\u0414\u0435\u043c\u043e-\u0437\u0430\u043d\u044f\u0442\u0438\u0435 \u043a\u0443\u0440\u0441\u0430 \u00abDataOps Engineer\u00bb\" width=\"770\" height=\"433\" src=\"https:\/\/www.youtube.com\/embed\/CW9CkG40bPM?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"\u0414\u0435\u043c\u043e-\u0437\u0430\u043d\u044f\u0442\u0438\u0435 \u043a\u0443\u0440\u0441\u0430 \u00abDataOps Engineer\u00bb\" width=\"770\" height=\"433\" src=\"https:\/\/www.youtube.com\/embed\/O0XqUVuU3bA?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>\u041f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 Data Pipelines \u0432 Apache Airflow \u042d\u0442\u0430 \u0441\u0442\u0430\u0442\u044c\u044f \u043f\u043e\u0441\u0432\u044f\u0449\u0435\u043d\u0430 \u043a\u0440\u0430\u0442\u043a\u043e\u043c\u0443 \u0432\u0432\u0435\u0434\u0435\u043d\u0438\u044e \u0432 Airflow \u0438 \u0448\u0430\u0433\u043e\u0432 \u043f\u043e \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044e \u0438 \u043d\u0430\u0441\u0442\u0440\u043e\u0439\u043a\u0435 \u043a\u043e\u043d\u0432\u0435\u0439\u0435\u0440\u043e\u0432 \u0434\u0430\u043d\u043d\u044b\u0445 (Data Pipelines). \u0421\u043d\u0430\u0447\u0430\u043b\u0430 \u043c\u044b \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u043c \u0438 \u0441\u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u0443\u0435\u043c Airflow. \u0417\u0430\u0442\u0435\u043c \u0440\u0430\u0441\u0441\u043c\u043e\u0442\u0440\u0438\u043c \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u044f \u0438 \u0437\u0430\u043f\u0443\u0441\u043a\u0430 DAG \u0432 Airflow. \u041d\u0430\u0448\u0435\u0439 \u0446\u0435\u043b\u044c\u044e \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f \u043f\u0440\u0430\u043a\u0442\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u043f\u043e\u043d\u0438\u043c\u0430\u043d\u0438\u0435 \u0440\u0430\u0437\u0432\u0435\u0440\u0442\u044b\u0432\u0430\u043d\u0438\u044f Airflow \u0438 \u0431\u0430\u0437\u043e\u0432\u043e\u0439 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u043a\u0438 DAG. \u041a\u0430\u043a \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c \u043e\u0431\u044a\u0435\u043a\u0442\u044b Avro \u0441<\/p>\n","protected":false},"author":10,"featured_media":4282,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[7],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/4281"}],"collection":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/comments?post=4281"}],"version-history":[{"count":1,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/4281\/revisions"}],"predecessor-version":[{"id":4283,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/4281\/revisions\/4283"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/media\/4282"}],"wp:attachment":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/media?parent=4281"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/categories?post=4281"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/tags?post=4281"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}