{"id":9859,"date":"2026-09-30T09:00:00","date_gmt":"2026-09-30T09:00:00","guid":{"rendered":"https:\/\/otus.ru\/journal\/?p=9859"},"modified":"2026-09-30T15:52:57","modified_gmt":"2026-09-30T15:52:57","slug":"parser-opisanie-i-primer-koda","status":"publish","type":"post","link":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/","title":{"rendered":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430"},"content":{"rendered":"<p><strong>\u041f\u0430\u0440\u0441\u0435\u0440<\/strong> &#8212; \u044d\u0442\u043e \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u0430, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u0442 \u0434\u0430\u043d\u043d\u044b\u0435 \u043f\u043e \u0438\u0437\u0432\u0435\u0441\u0442\u043d\u044b\u043c \u043f\u0440\u0430\u0432\u0438\u043b\u0430\u043c \u0438 \u0434\u043e\u0441\u0442\u0430\u0435\u0442 \u0438\u0437 \u043d\u0438\u0445 \u043d\u0443\u0436\u043d\u044b\u0435 \u0447\u0430\u0441\u0442\u0438; \u0432 \u043e\u0431\u0438\u0445\u043e\u0434\u0435 \u0442\u0430\u043a \u0447\u0430\u0449\u0435 \u0432\u0441\u0435\u0433\u043e \u043d\u0430\u0437\u044b\u0432\u0430\u044e\u0442 \u0441\u043a\u0440\u0438\u043f\u0442, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u0432\u0435\u0431-\u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u043d\u0430\u0445\u043e\u0434\u0438\u0442 \u0432 HTML \u043d\u0443\u0436\u043d\u044b\u0435 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u044b \u0438 \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u0435\u0442 \u0438\u0445 \u0432 \u0442\u0430\u0431\u043b\u0438\u0446\u0443. \u041d\u0438\u0436\u0435 &#8212; \u0440\u0430\u0431\u043e\u0447\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u0442\u0430\u043a\u043e\u0433\u043e \u043f\u0430\u0440\u0441\u0435\u0440\u0430 \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML, \u0437\u0430\u043f\u0438\u0441\u044c \u0432 CSV, \u0443\u0432\u0430\u0436\u0435\u043d\u0438\u0435 \u043a <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>, \u043f\u0430\u0443\u0437\u044b \u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043e\u0448\u0438\u0431\u043e\u043a.<\/p>\n<div class=\"oj-toc\" style=\"background:#f6f6f6;border-radius:8px;padding:14px 20px;margin:20px 0\">\n<p><strong>\u0421\u043e\u0434\u0435\u0440\u0436\u0430\u043d\u0438\u0435<\/strong><\/p>\n<ol>\n<li><a href=\"#s1\">\u0418\u0437 \u0447\u0435\u0433\u043e \u0441\u043e\u0441\u0442\u043e\u0438\u0442 \u043f\u0430\u0440\u0441\u0435\u0440 \u0441\u0430\u0439\u0442\u0430<\/a><\/li>\n<li><a href=\"#s2\">\u0423\u0447\u0435\u0431\u043d\u044b\u0439 \u0441\u0442\u0435\u043d\u0434: \u0441\u0430\u0439\u0442 \u043d\u0430 \u0441\u0432\u043e\u0435\u043c \u043a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u0435<\/a><\/li>\n<li><a href=\"#s3\">\u041f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u043f\u0430\u0440\u0441\u0435\u0440\u0430 \u0446\u0435\u043b\u0438\u043a\u043e\u043c<\/a><\/li>\n<li><a href=\"#s4\">\u0420\u0430\u0437\u0431\u043e\u0440 \u043f\u043e \u0448\u0430\u0433\u0430\u043c<\/a><\/li>\n<li><a href=\"#s5\">\u0427\u0430\u0441\u0442\u0430\u044f \u043e\u0448\u0438\u0431\u043a\u0430: \u043a\u0440\u0430\u043a\u043e\u0437\u044f\u0431\u0440\u044b \u0432\u043c\u0435\u0441\u0442\u043e \u0440\u0443\u0441\u0441\u043a\u043e\u0433\u043e \u0442\u0435\u043a\u0441\u0442\u0430<\/a><\/li>\n<li><a href=\"#s6\">\u0412\u0435\u0436\u043b\u0438\u0432\u043e\u0441\u0442\u044c \u0438 \u0431\u0435\u0437\u043e\u043f\u0430\u0441\u043d\u043e\u0441\u0442\u044c<\/a><\/li>\n<li><a href=\"#s7\">\u0415\u0441\u043b\u0438 \u043d\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c<\/a><\/li>\n<li><a href=\"#s8\">\u0412\u044b\u0432\u043e\u0434\u044b<\/a><\/li>\n<li><a href=\"#s9\">\u0413\u0434\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442\u0441\u044f \/ \u0441\u0432\u044f\u0437\u044c \u0441 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u043e\u0439<\/a><\/li>\n<li><a href=\"#s10\">FAQ<\/a><\/li>\n<\/ol>\n<\/div>\n<p>\u0422\u0440\u0438 \u0441\u043b\u043e\u0432\u0430, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0447\u0430\u0441\u0442\u043e \u0441\u043c\u0435\u0448\u0438\u0432\u0430\u044e\u0442:<\/p>\n<ul>\n<li><strong>\u043f\u0430\u0440\u0441\u0438\u043d\u0433<\/strong> &#8212; \u0440\u0430\u0437\u0431\u043e\u0440 \u0443\u0436\u0435 \u043f\u043e\u043b\u0443\u0447\u0435\u043d\u043d\u043e\u0433\u043e \u0442\u0435\u043a\u0441\u0442\u0430 (HTML, JSON, CSV) \u043d\u0430 \u0441\u0442\u0440\u0443\u043a\u0442\u0443\u0440\u0443;<\/li>\n<li><strong>\u0441\u043a\u0440\u0430\u043f\u0438\u043d\u0433<\/strong> &#8212; \u0441\u0431\u043e\u0440 \u0434\u0430\u043d\u043d\u044b\u0445 \u0441 \u0441\u0430\u0439\u0442\u043e\u0432 \u0446\u0435\u043b\u0438\u043a\u043e\u043c: \u0437\u0430\u0433\u0440\u0443\u0437\u043a\u0430 \u0441\u0442\u0440\u0430\u043d\u0438\u0446 \u043f\u043b\u044e\u0441 \u0438\u0445 \u043f\u0430\u0440\u0441\u0438\u043d\u0433;<\/li>\n<li><strong>\u043a\u0440\u0430\u0443\u043b\u0438\u043d\u0433<\/strong> &#8212; \u043e\u0431\u0445\u043e\u0434 \u0441\u0441\u044b\u043b\u043e\u043a, \u0447\u0442\u043e\u0431\u044b \u043d\u0430\u0439\u0442\u0438 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b \u0434\u043b\u044f \u0441\u0431\u043e\u0440\u0430.<\/li>\n<\/ul>\n<p>\u0421\u043a\u0440\u0438\u043f\u0442 \u0438\u0437 \u0441\u0442\u0430\u0442\u044c\u0438 \u0434\u0435\u043b\u0430\u0435\u0442 \u0432\u0441\u0435 \u0442\u0440\u0438 \u0448\u0430\u0433\u0430 \u0432 \u043c\u0438\u043d\u0438\u0430\u0442\u044e\u0440\u0435, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0434\u0430\u043b\u044c\u0448\u0435 \u043e\u043d \u043d\u0430\u0437\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u043f\u0430\u0440\u0441\u0435\u0440\u043e\u043c, \u043a\u0430\u043a \u043f\u0440\u0438\u043d\u044f\u0442\u043e \u0432 \u0440\u0430\u0437\u0433\u043e\u0432\u043e\u0440\u043d\u043e\u0439 \u0440\u0435\u0447\u0438.<\/p>\n<p>\u041a\u043e\u0434 \u043f\u0440\u043e\u0432\u0435\u0440\u0435\u043d 30 \u0441\u0435\u043d\u0442\u044f\u0431\u0440\u044f 2026 \u0433\u043e\u0434\u0430 \u043d\u0430 Python 3.14.7 \u0441 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a\u0430\u043c\u0438 requests 2.34.2, beautifulsoup4 4.15.0 \u0438 lxml 6.1.3.<\/p>\n<h2 id=\"s1\">\u0418\u0437 \u0447\u0435\u0433\u043e \u0441\u043e\u0441\u0442\u043e\u0438\u0442 \u043f\u0430\u0440\u0441\u0435\u0440 \u0441\u0430\u0439\u0442\u0430<\/h2>\n<table>\n<thead>\n<tr>\n<th>\u0428\u0430\u0433<\/th>\n<th>\u0427\u0442\u043e \u0434\u0435\u043b\u0430\u0435\u0442<\/th>\n<th>\u0427\u0435\u043c \u0432 \u043f\u0440\u0438\u043c\u0435\u0440\u0435<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u041f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 \u043f\u0440\u0430\u0432\u0438\u043b<\/td>\n<td>\u0447\u0438\u0442\u0430\u0435\u0442 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code> \u0441\u0430\u0439\u0442\u0430<\/td>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">urllib.robotparser<\/code> \u0438\u0437 \u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u043e\u0439 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a\u0438<\/td>\n<\/tr>\n<tr>\n<td>\u0417\u0430\u0433\u0440\u0443\u0437\u043a\u0430<\/td>\n<td>\u043e\u0442\u043f\u0440\u0430\u0432\u043b\u044f\u0435\u0442 HTTP-\u0437\u0430\u043f\u0440\u043e\u0441 GET, \u043f\u043e\u043b\u0443\u0447\u0430\u0435\u0442 HTML<\/td>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests<\/code><\/td>\n<\/tr>\n<tr>\n<td>\u0420\u0430\u0437\u0431\u043e\u0440<\/td>\n<td>\u0441\u0442\u0440\u043e\u0438\u0442 \u0438\u0437 HTML \u0434\u0435\u0440\u0435\u0432\u043e \u0438 \u0438\u0449\u0435\u0442 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u044b<\/td>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">BeautifulSoup<\/code> \u043f\u043e\u0432\u0435\u0440\u0445 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">lxml<\/code><\/td>\n<\/tr>\n<tr>\n<td>\u041e\u0447\u0438\u0441\u0442\u043a\u0430<\/td>\n<td>\u043f\u0440\u0438\u0432\u043e\u0434\u0438\u0442 \u0442\u0435\u043a\u0441\u0442 \u043a \u0442\u0438\u043f\u0430\u043c: \u0446\u0435\u043d\u0430 \u0441\u0442\u0440\u043e\u043a\u043e\u0439 -&gt; \u0447\u0438\u0441\u043b\u043e<\/td>\n<td>\u0441\u0432\u043e\u0438 \u0444\u0443\u043d\u043a\u0446\u0438\u0438<\/td>\n<\/tr>\n<tr>\n<td>\u0421\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435<\/td>\n<td>\u043f\u0438\u0448\u0435\u0442 \u0441\u0442\u0440\u043e\u043a\u0438 \u0432 \u0444\u0430\u0439\u043b<\/td>\n<td>\u043c\u043e\u0434\u0443\u043b\u044c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">csv<\/code><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Beautiful Soup \u0441\u0430\u043c HTML \u043d\u0435 \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u0442: \u043e\u043d \u0434\u0430\u0435\u0442 \u0443\u0434\u043e\u0431\u043d\u044b\u0439 \u043f\u043e\u0438\u0441\u043a \u043f\u043e \u0434\u0435\u0440\u0435\u0432\u0443, \u0430 \u0434\u0435\u0440\u0435\u0432\u043e \u0441\u0442\u0440\u043e\u0438\u0442 \u043f\u043e\u0434\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u0439 \u0440\u0430\u0437\u0431\u043e\u0440\u0449\u0438\u043a. \u0412\u0430\u0440\u0438\u0430\u043d\u0442\u043e\u0432 \u0442\u0440\u0438.<\/p>\n<table>\n<thead>\n<tr>\n<th>\u0420\u0430\u0437\u0431\u043e\u0440\u0449\u0438\u043a<\/th>\n<th>\u0423\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0430<\/th>\n<th>\u041a\u043e\u0433\u0434\u0430 \u0431\u0440\u0430\u0442\u044c<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">html.parser<\/code><\/td>\n<td>\u0432\u0441\u0442\u0440\u043e\u0435\u043d \u0432 Python<\/td>\n<td>\u043d\u0435\u0442 \u0432\u043e\u0437\u043c\u043e\u0436\u043d\u043e\u0441\u0442\u0438 \u0441\u0442\u0430\u0432\u0438\u0442\u044c \u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e\u0441\u0442\u0438<\/td>\n<\/tr>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">lxml<\/code><\/td>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">pip install lxml<\/code><\/td>\n<td>\u043e\u0431\u044b\u0447\u043d\u044b\u0439 \u0432\u044b\u0431\u043e\u0440: \u0431\u044b\u0441\u0442\u0440\u044b\u0439 \u0438 \u0442\u0435\u0440\u043f\u0438\u043c\u044b\u0439 \u043a \u043a\u0440\u0438\u0432\u043e\u0439 \u0440\u0430\u0437\u043c\u0435\u0442\u043a\u0435<\/td>\n<\/tr>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">html5lib<\/code><\/td>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">pip install html5lib<\/code><\/td>\n<td>\u043d\u0443\u0436\u043d\u043e \u0434\u0435\u0440\u0435\u0432\u043e, \u043a\u0430\u043a \u0435\u0433\u043e \u043f\u043e\u0441\u0442\u0440\u043e\u0438\u043b \u0431\u044b \u0431\u0440\u0430\u0443\u0437\u0435\u0440; \u0437\u0430\u043c\u0435\u0442\u043d\u043e \u043c\u0435\u0434\u043b\u0435\u043d\u043d\u0435\u0435<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u041d\u0430 \u0430\u043a\u043a\u0443\u0440\u0430\u0442\u043d\u043e\u0439 \u0440\u0430\u0437\u043c\u0435\u0442\u043a\u0435 \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442 \u043e\u0434\u0438\u043d\u0430\u043a\u043e\u0432\u044b\u0439, \u043d\u0430 \u0441\u043b\u043e\u043c\u0430\u043d\u043d\u043e\u0439 (\u043d\u0435\u0437\u0430\u043a\u0440\u044b\u0442\u044b\u0435 \u0442\u0435\u0433\u0438) \u0434\u0435\u0440\u0435\u0432\u044c\u044f \u043c\u043e\u0433\u0443\u0442 \u043e\u0442\u043b\u0438\u0447\u0430\u0442\u044c\u0441\u044f. \u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u0440\u0430\u0437\u0431\u043e\u0440\u0449\u0438\u043a \u0443\u043a\u0430\u0437\u044b\u0432\u0430\u044e\u0442 \u044f\u0432\u043d\u043e, \u0432\u0442\u043e\u0440\u044b\u043c \u0430\u0440\u0433\u0443\u043c\u0435\u043d\u0442\u043e\u043c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">BeautifulSoup<\/code>.<\/p>\n<h2 id=\"s2\">\u0423\u0447\u0435\u0431\u043d\u044b\u0439 \u0441\u0442\u0435\u043d\u0434: \u0441\u0430\u0439\u0442 \u043d\u0430 \u0441\u0432\u043e\u0435\u043c \u043a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u0435<\/h2>\n<p>\u0423\u0447\u0438\u0442\u044c\u0441\u044f \u043d\u0430 \u0447\u0443\u0436\u043e\u043c \u0441\u0430\u0439\u0442\u0435 \u043d\u0435\u0443\u0434\u043e\u0431\u043d\u043e: \u0440\u0430\u0437\u043c\u0435\u0442\u043a\u0430 \u043c\u0435\u043d\u044f\u0435\u0442\u0441\u044f, \u0430 \u0447\u0430\u0441\u0442\u044b\u0435 \u0437\u0430\u043f\u0440\u043e\u0441\u044b \u043c\u0435\u0448\u0430\u044e\u0442 \u0432\u043b\u0430\u0434\u0435\u043b\u044c\u0446\u0443. \u041f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u0440\u0438\u043c\u0435\u0440 \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u0442 \u0441 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u043c \u043a\u0430\u0442\u0430\u043b\u043e\u0433\u043e\u043c \u043a\u043d\u0438\u0433, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043f\u043e\u0434\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043b\u043e\u043a\u0430\u043b\u044c\u043d\u043e. \u0421\u043e\u0437\u0434\u0430\u0439\u0442\u0435 \u043d\u043e\u0432\u0443\u044e \u043f\u0443\u0441\u0442\u0443\u044e \u043f\u0430\u043f\u043a\u0443 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">parser-demo<\/code>, \u0432 \u043d\u0435\u0439 \u0432\u0438\u0440\u0442\u0443\u0430\u043b\u044c\u043d\u043e\u0435 \u043e\u043a\u0440\u0443\u0436\u0435\u043d\u0438\u0435, \u0438 \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u0435 \u0431\u0438\u0431\u043b\u0438\u043e\u0442\u0435\u043a\u0438:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\">python<span style=\"color: #ABB2BF\"> <\/span>-m<span style=\"color: #ABB2BF\"> <\/span>pip<span style=\"color: #ABB2BF\"> <\/span>install<span style=\"color: #ABB2BF\"> <\/span>requests<span style=\"color: #ABB2BF\"> <\/span>beautifulsoup4<span style=\"color: #ABB2BF\"> <\/span>lxml<\/code><\/pre>\n<p>\u0424\u0430\u0439\u043b <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">make_site.py<\/code> \u0441\u043e\u0437\u0434\u0430\u0435\u0442 \u0432 \u043f\u043e\u0434\u043f\u0430\u043f\u043a\u0435 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">site<\/code> \u0442\u0440\u0438 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b \u0438 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\"><span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">html<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">escape<\/span>\n<span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">pathlib<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">Path<\/span>\n\n<span style=\"color: #E06C75\">site<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">Path<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;site&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n<span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">site<\/span> <span style=\"color: #56B6C2\">\/<\/span> <span style=\"color: #98C379\">&quot;private&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">mkdir<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">parents<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">True<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">exist_ok<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">True<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n<span style=\"color: #E06C75\">PAGE<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">&quot;&quot;&quot;&lt;!doctype html&gt;<\/span>\n<span style=\"color: #98C379\">&lt;html lang=&quot;ru&quot;&gt;&lt;head&gt;&lt;meta charset=&quot;utf-8&quot;&gt;&lt;title&gt;\u041a\u0430\u0442\u0430\u043b\u043e\u0433&lt;\/title&gt;&lt;\/head&gt;<\/span>\n<span style=\"color: #98C379\">&lt;body&gt;<\/span>\n<span style=\"color: #98C379\">{cards}<\/span>\n<span style=\"color: #98C379\">{nav}<\/span>\n<span style=\"color: #98C379\">&lt;\/body&gt;&lt;\/html&gt;<\/span>\n<span style=\"color: #98C379\">&quot;&quot;&quot;<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">book_id<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">title<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">price<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #E06C75\">price_html<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">f&#39;&lt;span class=&quot;price&quot;&gt;{<\/span><span style=\"color: #E06C75\">price<\/span><span style=\"color: #98C379\">}&lt;\/span&gt;&#39;<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">price<\/span> <span style=\"color: #C678DD\">else<\/span> <span style=\"color: #98C379\">&quot;&quot;<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&#39;&lt;article class=&quot;book&quot;&gt;&lt;h2&gt;&lt;a href=&quot;\/book\/{<\/span><span style=\"color: #E06C75\">book_id<\/span><span style=\"color: #98C379\">}&quot;&gt;&#39;<\/span>\n            <span style=\"color: #98C379\">f&quot;{<\/span><span style=\"color: #E06C75\">escape<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">title<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #98C379\">}&lt;\/a&gt;&lt;\/h2&gt;{<\/span><span style=\"color: #E06C75\">price_html<\/span><span style=\"color: #98C379\">}&lt;\/article&gt;&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">page<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">cards<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">next_href<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #E06C75\">nav<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">f&#39;&lt;a class=&quot;next&quot; href=&quot;{<\/span><span style=\"color: #E06C75\">next_href<\/span><span style=\"color: #98C379\">}&quot;&gt;\u0414\u0430\u043b\u044c\u0448\u0435&lt;\/a&gt;&#39;<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">next_href<\/span> <span style=\"color: #C678DD\">else<\/span> <span style=\"color: #98C379\">&quot;&quot;<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E06C75\">PAGE<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">format<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">cards<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #98C379\">&quot;\\n&quot;<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">join<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">cards<\/span><span style=\"color: #ABB2BF\">),<\/span> <span style=\"color: #E06C75\">nav<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E06C75\">nav<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n\n<span style=\"color: #E06C75\">pages<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #ABB2BF\">{<\/span>\n    <span style=\"color: #98C379\">&quot;index.html&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">page<\/span><span style=\"color: #ABB2BF\">([<\/span><span style=\"color: #E06C75\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">1<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\u0418\u0437\u0443\u0447\u0430\u0435\u043c Python&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;1&amp;nbsp;290 \u20bd&quot;<\/span><span style=\"color: #ABB2BF\">),<\/span>\n                        <span style=\"color: #E06C75\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">2<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\u0427\u0438\u0441\u0442\u044b\u0439 \u043a\u043e\u0434&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;990 \u20bd&quot;<\/span><span style=\"color: #ABB2BF\">)],<\/span> <span style=\"color: #98C379\">&quot;page2.html&quot;<\/span><span style=\"color: #ABB2BF\">),<\/span>\n    <span style=\"color: #98C379\">&quot;page2.html&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">page<\/span><span style=\"color: #ABB2BF\">([<\/span><span style=\"color: #E06C75\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">3<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;=1+1 \u0438 \u0434\u0440\u0443\u0433\u0438\u0435 \u0444\u043e\u0440\u043c\u0443\u043b\u044b&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;450 \u20bd&quot;<\/span><span style=\"color: #ABB2BF\">),<\/span>\n                        <span style=\"color: #E06C75\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">4<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\u041a\u043d\u0438\u0433\u0430 \u0431\u0435\u0437 \u0446\u0435\u043d\u044b&quot;<\/span><span style=\"color: #ABB2BF\">)],<\/span> <span style=\"color: #98C379\">&quot;\/private\/page3.html&quot;<\/span><span style=\"color: #ABB2BF\">),<\/span>\n    <span style=\"color: #98C379\">&quot;private\/page3.html&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">page<\/span><span style=\"color: #ABB2BF\">([<\/span><span style=\"color: #E06C75\">card<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">5<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\u0417\u0430\u043a\u0440\u044b\u0442\u044b\u0439 \u0440\u0430\u0437\u0434\u0435\u043b&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;100 \u20bd&quot;<\/span><span style=\"color: #ABB2BF\">)]),<\/span>\n    <span style=\"color: #98C379\">&quot;robots.txt&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #98C379\">&quot;User-agent: *\\nDisallow: \/private\/\\nCrawl-delay: 1\\n&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span>\n<span style=\"color: #ABB2BF\">}<\/span>\n<span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">name<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">text<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">pages<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">items<\/span><span style=\"color: #ABB2BF\">():<\/span>\n    <span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">site<\/span> <span style=\"color: #56B6C2\">\/<\/span> <span style=\"color: #E06C75\">name<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">write_text<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">text<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">encoding<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #98C379\">&quot;utf-8&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n<span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;\u0441\u043e\u0437\u0434\u0430\u043d\u043e \u0444\u0430\u0439\u043b\u043e\u0432:&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E5C07B\">len<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">pages<\/span><span style=\"color: #ABB2BF\">))<\/span><\/code><\/pre>\n<p>\u0412 \u043a\u0430\u0442\u0430\u043b\u043e\u0433\u0435 \u043d\u0430\u0440\u043e\u0447\u043d\u043e \u0437\u0430\u043b\u043e\u0436\u0435\u043d\u044b \u043d\u0435\u0443\u0434\u043e\u0431\u043d\u044b\u0435 \u0441\u043b\u0443\u0447\u0430\u0438: \u0446\u0435\u043d\u0430 \u0441 \u043d\u0435\u0440\u0430\u0437\u0440\u044b\u0432\u043d\u044b\u043c \u043f\u0440\u043e\u0431\u0435\u043b\u043e\u043c, \u043a\u043d\u0438\u0433\u0430 \u0431\u0435\u0437 \u0446\u0435\u043d\u044b, \u043d\u0430\u0437\u0432\u0430\u043d\u0438\u0435, \u043a\u043e\u0442\u043e\u0440\u043e\u0435 \u043d\u0430\u0447\u0438\u043d\u0430\u0435\u0442\u0441\u044f \u0441\u043e \u0437\u043d\u0430\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">=<\/code>, \u0438 \u0442\u0440\u0435\u0442\u044c\u044f \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u0432 \u0440\u0430\u0437\u0434\u0435\u043b\u0435, \u0437\u0430\u043a\u0440\u044b\u0442\u043e\u043c \u0432 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>. \u0417\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u0435 \u0441\u043a\u0440\u0438\u043f\u0442, \u0430 \u0437\u0430\u0442\u0435\u043c \u0432 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e\u043c \u043e\u043a\u043d\u0435 \u0442\u0435\u0440\u043c\u0438\u043d\u0430\u043b\u0430 &#8212; \u0432\u0441\u0442\u0440\u043e\u0435\u043d\u043d\u044b\u0439 \u0432\u0435\u0431-\u0441\u0435\u0440\u0432\u0435\u0440 Python, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u044b\u0439 \u0442\u043e\u043b\u044c\u043a\u043e \u0441 \u044d\u0442\u043e\u0433\u043e \u0436\u0435 \u043a\u043e\u043c\u043f\u044c\u044e\u0442\u0435\u0440\u0430:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\">python<span style=\"color: #ABB2BF\"> <\/span>make_site.py\npython<span style=\"color: #ABB2BF\"> <\/span>-m<span style=\"color: #ABB2BF\"> <\/span>http.server<span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #D19A66\">8000<\/span><span style=\"color: #ABB2BF\"> <\/span>--bind<span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #D19A66\">127<\/span>.0.0.1<span style=\"color: #ABB2BF\"> <\/span>--directory<span style=\"color: #ABB2BF\"> <\/span>site<\/code><\/pre>\n<h2 id=\"s3\">\u041f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u043f\u0430\u0440\u0441\u0435\u0440\u0430 \u0446\u0435\u043b\u0438\u043a\u043e\u043c<\/h2>\n<p>\u0424\u0430\u0439\u043b <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">parser.py<\/code> \u0432 \u0442\u043e\u0439 \u0436\u0435 \u043f\u0430\u043f\u043a\u0435 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">parser-demo<\/code>:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\"><span style=\"color: #C678DD\">import<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">csv<\/span>\n<span style=\"color: #C678DD\">import<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">time<\/span>\n<span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">urllib.parse<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">urljoin<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">urlsplit<\/span>\n<span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">urllib.robotparser<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">RobotFileParser<\/span>\n\n<span style=\"color: #C678DD\">import<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">requests<\/span>\n<span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">bs4<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">BeautifulSoup<\/span>\n\n<span style=\"color: #E06C75\">START_URL<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">&quot;http:\/\/127.0.0.1:8000\/index.html&quot;<\/span>\n<span style=\"color: #E06C75\">USER_AGENT<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">&quot;demo-parser\/1.0 (learning example)&quot;<\/span>\n<span style=\"color: #E06C75\">MAX_PAGES<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #D19A66\">5<\/span>\n<span style=\"color: #E06C75\">DEFAULT_DELAY<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #D19A66\">1.0<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">fetch<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">session<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #C678DD\">try<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E06C75\">response<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">session<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">timeout<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">5<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #D19A66\">15<\/span><span style=\"color: #ABB2BF\">),<\/span> <span style=\"color: #E06C75\">allow_redirects<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">False<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">raise_for_status<\/span><span style=\"color: #ABB2BF\">()<\/span>\n    <span style=\"color: #C678DD\">except<\/span> <span style=\"color: #E06C75\">requests<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">RequestException<\/span> <span style=\"color: #C678DD\">as<\/span> <span style=\"color: #E06C75\">error<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;\u043d\u0435 \u0437\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043e {<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #98C379\">}: {<\/span><span style=\"color: #E5C07B\">type<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">error<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">__name__<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E5C07B\">None<\/span>\n    <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">is_redirect<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;\u0441\u0442\u043e\u043f, \u043f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435: {<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E5C07B\">None<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E06C75\">response<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">load_robots<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">session<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">start_url<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #E06C75\">robots_url<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">urljoin<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">start_url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\/robots.txt&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #E06C75\">robots<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">RobotFileParser<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">robots_url<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #C678DD\">try<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E06C75\">response<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">session<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">robots_url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">timeout<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">5<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #D19A66\">15<\/span><span style=\"color: #ABB2BF\">))<\/span>\n    <span style=\"color: #C678DD\">except<\/span> <span style=\"color: #E06C75\">requests<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">RequestException<\/span> <span style=\"color: #C678DD\">as<\/span> <span style=\"color: #E06C75\">error<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;robots.txt \u043d\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d: {<\/span><span style=\"color: #E5C07B\">type<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">error<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">__name__<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E5C07B\">None<\/span>\n    <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">status_code<\/span> <span style=\"color: #56B6C2\">==<\/span> <span style=\"color: #D19A66\">404<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E06C75\">robots<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">parse<\/span><span style=\"color: #ABB2BF\">([])<\/span>\n    <span style=\"color: #C678DD\">elif<\/span> <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">ok<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E06C75\">robots<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">parse<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">text<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">splitlines<\/span><span style=\"color: #ABB2BF\">())<\/span>\n    <span style=\"color: #C678DD\">else<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;robots.txt \u0432\u0435\u0440\u043d\u0443\u043b \u043a\u043e\u0434 {<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">status_code<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E5C07B\">None<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E06C75\">robots<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">to_int<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">text<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #E06C75\">digits<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #98C379\">&quot;&quot;<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">join<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">ch<\/span> <span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">ch<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">text<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">ch<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #98C379\">&quot;0123456789&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E5C07B\">int<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">digits<\/span><span style=\"color: #ABB2BF\">)<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">digits<\/span> <span style=\"color: #C678DD\">else<\/span> <span style=\"color: #E5C07B\">None<\/span>\n\n\n<span style=\"color: #E06C75\">DANGEROUS_FIRST<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;=&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;+&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;-&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;@&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\\t&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\\r&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\\n&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\uff1d&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\uff0b&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\uff0d&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;\uff20&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">safe_cell<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">value<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E5C07B\">isinstance<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">value<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E5C07B\">str<\/span><span style=\"color: #ABB2BF\">)<\/span> <span style=\"color: #56B6C2\">and<\/span> <span style=\"color: #E06C75\">value<\/span><span style=\"color: #ABB2BF\">[:<\/span><span style=\"color: #D19A66\">1<\/span><span style=\"color: #ABB2BF\">]<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">DANGEROUS_FIRST<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #98C379\">&quot;&#39;&quot;<\/span> <span style=\"color: #56B6C2\">+<\/span> <span style=\"color: #E06C75\">value<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E06C75\">value<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">parse_books<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">html<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">page_url<\/span><span style=\"color: #ABB2BF\">):<\/span>\n    <span style=\"color: #E06C75\">soup<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">BeautifulSoup<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">html<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;lxml&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #E06C75\">rows<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #ABB2BF\">[]<\/span>\n    <span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">card<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">soup<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;article.book&quot;<\/span><span style=\"color: #ABB2BF\">):<\/span>\n        <span style=\"color: #E06C75\">link<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">card<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select_one<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;h2 a&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">link<\/span> <span style=\"color: #56B6C2\">is<\/span> <span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">:<\/span>\n            <span style=\"color: #C678DD\">continue<\/span>\n        <span style=\"color: #E06C75\">price<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">card<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select_one<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;.price&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #E06C75\">rows<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">append<\/span><span style=\"color: #ABB2BF\">({<\/span>\n            <span style=\"color: #98C379\">&quot;title&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">link<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get_text<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">strip<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #E5C07B\">True<\/span><span style=\"color: #ABB2BF\">),<\/span>\n            <span style=\"color: #98C379\">&quot;price&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">to_int<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">price<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get_text<\/span><span style=\"color: #ABB2BF\">())<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">price<\/span> <span style=\"color: #C678DD\">else<\/span> <span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">,<\/span>\n            <span style=\"color: #98C379\">&quot;url&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">urljoin<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">page_url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">link<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;href&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;&quot;<\/span><span style=\"color: #ABB2BF\">)),<\/span>\n        <span style=\"color: #ABB2BF\">})<\/span>\n    <span style=\"color: #E06C75\">next_link<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">soup<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select_one<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;a.next&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #E06C75\">next_url<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">urljoin<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">page_url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">next_link<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;href&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;&quot;<\/span><span style=\"color: #ABB2BF\">))<\/span> <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">next_link<\/span> <span style=\"color: #C678DD\">else<\/span> <span style=\"color: #E5C07B\">None<\/span>\n    <span style=\"color: #C678DD\">return<\/span> <span style=\"color: #E06C75\">rows<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">next_url<\/span>\n\n\n<span style=\"color: #C678DD\">def<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #61AFEF; font-weight: bold\">main<\/span><span style=\"color: #ABB2BF\">():<\/span>\n    <span style=\"color: #E06C75\">session<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">requests<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">Session<\/span><span style=\"color: #ABB2BF\">()<\/span>\n    <span style=\"color: #E06C75\">session<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">headers<\/span><span style=\"color: #ABB2BF\">[<\/span><span style=\"color: #98C379\">&quot;User-Agent&quot;<\/span><span style=\"color: #ABB2BF\">]<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">USER_AGENT<\/span>\n\n    <span style=\"color: #E06C75\">robots<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">load_robots<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">session<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">START_URL<\/span><span style=\"color: #ABB2BF\">)<\/span>\n    <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">robots<\/span> <span style=\"color: #56B6C2\">is<\/span> <span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;\u0441\u0442\u043e\u043f: \u043f\u0440\u0430\u0432\u0438\u043b\u0430 \u0441\u0430\u0439\u0442\u0430 \u043d\u0435\u0438\u0437\u0432\u0435\u0441\u0442\u043d\u044b, \u043d\u0438\u0447\u0435\u0433\u043e \u043d\u0435 \u0437\u0430\u0433\u0440\u0443\u0436\u0430\u044e&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">return<\/span>\n    <span style=\"color: #E06C75\">delay<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">robots<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">crawl_delay<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">USER_AGENT<\/span><span style=\"color: #ABB2BF\">)<\/span> <span style=\"color: #56B6C2\">or<\/span> <span style=\"color: #E06C75\">DEFAULT_DELAY<\/span>\n\n    <span style=\"color: #E06C75\">rows<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">host<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #ABB2BF\">[],<\/span> <span style=\"color: #E06C75\">START_URL<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">urlsplit<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">START_URL<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">netloc<\/span>\n    <span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">_<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E5C07B\">range<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">MAX_PAGES<\/span><span style=\"color: #ABB2BF\">):<\/span>\n        <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">url<\/span> <span style=\"color: #56B6C2\">is<\/span> <span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">:<\/span>\n            <span style=\"color: #C678DD\">break<\/span>\n        <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">urlsplit<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">netloc<\/span> <span style=\"color: #56B6C2\">!=<\/span> <span style=\"color: #E06C75\">host<\/span><span style=\"color: #ABB2BF\">:<\/span>\n            <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;\u0441\u0442\u043e\u043f, \u0447\u0443\u0436\u043e\u0439 \u0445\u043e\u0441\u0442: {<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n            <span style=\"color: #C678DD\">break<\/span>\n        <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #56B6C2\">not<\/span> <span style=\"color: #E06C75\">robots<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">can_fetch<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">USER_AGENT<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">):<\/span>\n            <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;\u0441\u0442\u043e\u043f, \u0437\u0430\u043f\u0440\u0435\u0449\u0435\u043d\u043e robots.txt: {<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n            <span style=\"color: #C678DD\">break<\/span>\n        <span style=\"color: #E06C75\">response<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">fetch<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">session<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">response<\/span> <span style=\"color: #56B6C2\">is<\/span> <span style=\"color: #E5C07B\">None<\/span><span style=\"color: #ABB2BF\">:<\/span>\n            <span style=\"color: #C678DD\">break<\/span>\n        <span style=\"color: #E06C75\">page_rows<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">url<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">parse_books<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">content<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #E06C75\">rows<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">extend<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">page_rows<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;{<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">url<\/span><span style=\"color: #98C379\">}: \u043a\u0430\u0440\u0442\u043e\u0447\u0435\u043a {<\/span><span style=\"color: #E5C07B\">len<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">page_rows<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n        <span style=\"color: #E06C75\">time<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">sleep<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">delay<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n    <span style=\"color: #C678DD\">with<\/span> <span style=\"color: #E5C07B\">open<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;books.csv&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;w&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">newline<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #98C379\">&quot;&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">encoding<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #98C379\">&quot;utf-8-sig&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span> <span style=\"color: #C678DD\">as<\/span> <span style=\"color: #E06C75\">file<\/span><span style=\"color: #ABB2BF\">:<\/span>\n        <span style=\"color: #E06C75\">writer<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">csv<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">DictWriter<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">file<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">fieldnames<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #ABB2BF\">[<\/span><span style=\"color: #98C379\">&quot;title&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;price&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;url&quot;<\/span><span style=\"color: #ABB2BF\">])<\/span>\n        <span style=\"color: #E06C75\">writer<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">writeheader<\/span><span style=\"color: #ABB2BF\">()<\/span>\n        <span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">row<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">rows<\/span><span style=\"color: #ABB2BF\">:<\/span>\n            <span style=\"color: #E06C75\">writer<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">writerow<\/span><span style=\"color: #ABB2BF\">({<\/span><span style=\"color: #E06C75\">key<\/span><span style=\"color: #ABB2BF\">:<\/span> <span style=\"color: #E06C75\">safe_cell<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">value<\/span><span style=\"color: #ABB2BF\">)<\/span> <span style=\"color: #C678DD\">for<\/span> <span style=\"color: #E06C75\">key<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">value<\/span> <span style=\"color: #56B6C2\">in<\/span> <span style=\"color: #E06C75\">row<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">items<\/span><span style=\"color: #ABB2BF\">()})<\/span>\n    <span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">f&quot;\u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u043e \u0441\u0442\u0440\u043e\u043a: {<\/span><span style=\"color: #E5C07B\">len<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">rows<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #98C379\">}&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span>\n\n\n<span style=\"color: #C678DD\">if<\/span> <span style=\"color: #E06C75\">__name__<\/span> <span style=\"color: #56B6C2\">==<\/span> <span style=\"color: #98C379\">&quot;__main__&quot;<\/span><span style=\"color: #ABB2BF\">:<\/span>\n    <span style=\"color: #E06C75\">main<\/span><span style=\"color: #ABB2BF\">()<\/span><\/code><\/pre>\n<p>\u041a\u043e\u043c\u0430\u043d\u0434\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">python parser.py<\/code> \u043f\u0435\u0447\u0430\u0442\u0430\u0435\u0442 \u0445\u043e\u0434 \u0440\u0430\u0431\u043e\u0442\u044b:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\">http:\/\/127.0.0.1:8000\/index.html: \u043a\u0430\u0440\u0442\u043e\u0447\u0435\u043a 2\nhttp:\/\/127.0.0.1:8000\/page2.html: \u043a\u0430\u0440\u0442\u043e\u0447\u0435\u043a 2\n\u0441\u0442\u043e\u043f, \u0437\u0430\u043f\u0440\u0435\u0449\u0435\u043d\u043e robots.txt: http:\/\/127.0.0.1:8000\/private\/page3.html\n\u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u043e \u0441\u0442\u0440\u043e\u043a: 4\n<\/code><\/pre>\n<p>\u0412 \u043f\u0430\u043f\u043a\u0435 \u043f\u043e\u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">books.csv<\/code> (\u043f\u0440\u0438 \u043f\u043e\u0432\u0442\u043e\u0440\u043d\u043e\u043c \u0437\u0430\u043f\u0443\u0441\u043a\u0435 \u043e\u043d \u043f\u0435\u0440\u0435\u0437\u0430\u043f\u0438\u0441\u044b\u0432\u0430\u0435\u0442\u0441\u044f):<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\">title,price,url\n\u0418\u0437\u0443\u0447\u0430\u0435\u043c Python,1290,http:\/\/127.0.0.1:8000\/book\/1\n\u0427\u0438\u0441\u0442\u044b\u0439 \u043a\u043e\u0434,990,http:\/\/127.0.0.1:8000\/book\/2\n&#x27;=1+1 \u0438 \u0434\u0440\u0443\u0433\u0438\u0435 \u0444\u043e\u0440\u043c\u0443\u043b\u044b,450,http:\/\/127.0.0.1:8000\/book\/3\n\u041a\u043d\u0438\u0433\u0430 \u0431\u0435\u0437 \u0446\u0435\u043d\u044b,,http:\/\/127.0.0.1:8000\/book\/4\n<\/code><\/pre>\n<p>\u0421\u043e\u0431\u0440\u0430\u043d\u044b \u0447\u0435\u0442\u044b\u0440\u0435 \u043a\u043d\u0438\u0433\u0438 \u0441 \u0434\u0432\u0443\u0445 \u0441\u0442\u0440\u0430\u043d\u0438\u0446, \u0446\u0435\u043d\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">1 290 \u20bd<\/code> \u0441\u0442\u0430\u043b\u0430 \u0447\u0438\u0441\u043b\u043e\u043c 1290, \u0443 \u043a\u043d\u0438\u0433\u0438 \u0431\u0435\u0437 \u0446\u0435\u043d\u044b \u044f\u0447\u0435\u0439\u043a\u0430 \u043f\u0443\u0441\u0442\u0430\u044f, \u0430 \u0437\u0430\u043a\u0440\u044b\u0442\u0430\u044f \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u043d\u0435 \u0437\u0430\u043f\u0440\u0430\u0448\u0438\u0432\u0430\u043b\u0430\u0441\u044c.<\/p>\n<h2 id=\"s4\">\u0420\u0430\u0437\u0431\u043e\u0440 \u043f\u043e \u0448\u0430\u0433\u0430\u043c<\/h2>\n<p><strong>\u0417\u0430\u043f\u0440\u043e\u0441.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests.Session<\/code> \u043f\u0435\u0440\u0435\u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442 \u0441\u043e\u0435\u0434\u0438\u043d\u0435\u043d\u0438\u0435 \u0438 \u0445\u0440\u0430\u043d\u0438\u0442 \u043e\u0431\u0449\u0438\u0435 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0438. \u0412 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">User-Agent<\/code> \u0443\u043a\u0430\u0437\u0430\u043d\u043e \u0447\u0435\u0441\u0442\u043d\u043e\u0435 \u0438\u043c\u044f \u0441\u043a\u0440\u0438\u043f\u0442\u0430: \u0442\u0430\u043a \u0432\u043b\u0430\u0434\u0435\u043b\u0435\u0446 \u0441\u0430\u0439\u0442\u0430 \u0432\u0438\u0434\u0438\u0442 \u0432 \u043b\u043e\u0433\u0430\u0445, \u043a\u0442\u043e \u043a \u043d\u0435\u043c\u0443 \u0445\u043e\u0434\u0438\u0442. \u0423 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">timeout<\/code> \u0434\u0432\u0430 \u0447\u0438\u0441\u043b\u0430: \u0441\u0435\u043a\u0443\u043d\u0434\u044b \u043d\u0430 \u0441\u043e\u0435\u0434\u0438\u043d\u0435\u043d\u0438\u0435 \u0438 \u043d\u0430 \u043e\u0436\u0438\u0434\u0430\u043d\u0438\u0435 \u043e\u0447\u0435\u0440\u0435\u0434\u043d\u043e\u0439 \u043f\u043e\u0440\u0446\u0438\u0438 \u043e\u0442\u0432\u0435\u0442\u0430. \u0411\u0435\u0437 \u0442\u0430\u0439\u043c\u0430\u0443\u0442\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests<\/code> \u043c\u043e\u0436\u0435\u0442 \u0436\u0434\u0430\u0442\u044c \u0437\u0430\u0432\u0438\u0441\u0448\u0438\u0439 \u0441\u0435\u0440\u0432\u0435\u0440 \u0431\u0435\u0441\u043a\u043e\u043d\u0435\u0447\u043d\u043e. <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">raise_for_status()<\/code> \u043f\u0440\u0435\u0432\u0440\u0430\u0449\u0430\u0435\u0442 \u043e\u0442\u0432\u0435\u0442\u044b 4xx \u0438 5xx \u0432 \u0438\u0441\u043a\u043b\u044e\u0447\u0435\u043d\u0438\u0435.<\/p>\n<p><strong>\u0420\u0430\u0437\u0431\u043e\u0440 HTML.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">soup.select(\"article.book\")<\/code> \u043d\u0430\u0445\u043e\u0434\u0438\u0442 \u0432\u0441\u0435 \u043a\u0430\u0440\u0442\u043e\u0447\u043a\u0438 \u043f\u043e CSS-\u0441\u0435\u043b\u0435\u043a\u0442\u043e\u0440\u0443, \u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">card.select_one(...)<\/code> \u0438\u0449\u0435\u0442 \u0443\u0436\u0435 \u0432\u043d\u0443\u0442\u0440\u0438 \u043e\u0434\u043d\u043e\u0439 \u043a\u0430\u0440\u0442\u043e\u0447\u043a\u0438. \u0422\u0430\u043a \u043d\u0430\u0437\u0432\u0430\u043d\u0438\u0435 \u0438 \u0446\u0435\u043d\u0430 \u0433\u0430\u0440\u0430\u043d\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u043e \u043e\u0442\u043d\u043e\u0441\u044f\u0442\u0441\u044f \u043a \u043e\u0434\u043d\u043e\u0439 \u043a\u043d\u0438\u0433\u0435. \u0415\u0441\u043b\u0438 \u0441\u043e\u0431\u0440\u0430\u0442\u044c \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e \u0441\u043f\u0438\u0441\u043e\u043a \u043d\u0430\u0437\u0432\u0430\u043d\u0438\u0439 \u0438 \u043e\u0442\u0434\u0435\u043b\u044c\u043d\u043e \u0441\u043f\u0438\u0441\u043e\u043a \u0446\u0435\u043d, \u043a\u043d\u0438\u0433\u0430 \u0431\u0435\u0437 \u0446\u0435\u043d\u044b \u0441\u0434\u0432\u0438\u043d\u0435\u0442 \u043f\u0430\u0440\u044b. <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">select_one<\/code> \u0432\u043e\u0437\u0432\u0440\u0430\u0449\u0430\u0435\u0442 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">None<\/code>, \u043a\u043e\u0433\u0434\u0430 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u0430 \u043d\u0435\u0442, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u043f\u0435\u0440\u0435\u0434 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">.get_text()<\/code> \u0441\u0442\u043e\u0438\u0442 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0430.<\/p>\n<p><strong>\u041e\u0447\u0438\u0441\u0442\u043a\u0430.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">to_int<\/code> \u043e\u0441\u0442\u0430\u0432\u043b\u044f\u0435\u0442 \u0442\u043e\u043b\u044c\u043a\u043e \u0446\u0438\u0444\u0440\u044b ASCII. \u041c\u0435\u0442\u043e\u0434 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">str.isdigit()<\/code> \u0437\u0434\u0435\u0441\u044c \u043d\u0435 \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442: \u043e\u043d \u0441\u0447\u0438\u0442\u0430\u0435\u0442 \u0446\u0438\u0444\u0440\u043e\u0439 \u0438 \u0441\u0438\u043c\u0432\u043e\u043b <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\u00b2<\/code>, \u043d\u0430 \u043a\u043e\u0442\u043e\u0440\u043e\u043c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">int()<\/code> \u043f\u0430\u0434\u0430\u0435\u0442. \u0424\u0443\u043d\u043a\u0446\u0438\u044f \u0440\u0430\u0441\u0441\u0447\u0438\u0442\u0430\u043d\u0430 \u043d\u0430 \u0446\u0435\u043b\u044b\u0435 \u0446\u0435\u043d\u044b; \u0434\u043b\u044f \u0446\u0435\u043d \u0441 \u043a\u043e\u043f\u0435\u0439\u043a\u0430\u043c\u0438 \u0435\u0435 \u043d\u0443\u0436\u043d\u043e \u043f\u0435\u0440\u0435\u043f\u0438\u0441\u0430\u0442\u044c, \u0438\u043d\u0430\u0447\u0435 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">990.50<\/code> \u043f\u0440\u0435\u0432\u0440\u0430\u0442\u0438\u0442\u0441\u044f \u0432 99050.<\/p>\n<p><strong>\u0421\u0441\u044b\u043b\u043a\u0438.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">urljoin<\/code> \u0441\u043e\u0431\u0438\u0440\u0430\u0435\u0442 \u043f\u043e\u043b\u043d\u044b\u0439 \u0430\u0434\u0440\u0435\u0441 \u0438\u0437 \u043e\u0442\u043d\u043e\u0441\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0433\u043e <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">href<\/code>. \u0410\u0434\u0440\u0435\u0441 \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0435\u0439 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b \u043f\u0440\u0438\u0445\u043e\u0434\u0438\u0442 \u0438\u0437 \u0447\u0443\u0436\u043e\u0439 \u0440\u0430\u0437\u043c\u0435\u0442\u043a\u0438, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0446\u0438\u043a\u043b \u043f\u0440\u043e\u0432\u0435\u0440\u044f\u0435\u0442, \u0447\u0442\u043e \u0445\u043e\u0441\u0442 \u043d\u0435 \u0438\u0437\u043c\u0435\u043d\u0438\u043b\u0441\u044f, \u0438 \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">MAX_PAGES<\/code> &#8212; \u044d\u0442\u043e \u0437\u0430\u0449\u0438\u0442\u0430 \u043e\u0442 \u0431\u0435\u0441\u043a\u043e\u043d\u0435\u0447\u043d\u043e\u0433\u043e \u043e\u0431\u0445\u043e\u0434\u0430.<\/p>\n<p>\u041f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u044f \u0432 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">fetch<\/code> \u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u044b (<code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">allow_redirects=False<\/code>). \u0418\u043d\u0430\u0447\u0435 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests<\/code> \u0441\u0430\u043c \u043f\u0435\u0440\u0435\u0448\u0435\u043b \u0431\u044b \u043f\u043e \u0430\u0434\u0440\u0435\u0441\u0443 \u0438\u0437 \u043e\u0442\u0432\u0435\u0442\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u0430 &#8212; \u043d\u0430 \u0434\u0440\u0443\u0433\u043e\u0439 \u0445\u043e\u0441\u0442 \u0438\u043b\u0438 \u0432 \u0440\u0430\u0437\u0434\u0435\u043b, \u0437\u0430\u043a\u0440\u044b\u0442\u044b\u0439 \u0432 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>, &#8212; \u0443\u0436\u0435 \u043f\u043e\u0441\u043b\u0435 \u043e\u0431\u0435\u0438\u0445 \u043f\u0440\u043e\u0432\u0435\u0440\u043e\u043a. \u041d\u0430 \u0441\u0442\u0435\u043d\u0434\u0435 \u044d\u0442\u043e \u0432\u043e\u0441\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0441\u044f: \u0430\u0434\u0440\u0435\u0441 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\/private<\/code> \u0431\u0435\u0437 \u043a\u043e\u0441\u043e\u0439 \u0447\u0435\u0440\u0442\u044b \u043f\u0440\u0430\u0432\u0438\u043b\u043e\u043c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Disallow: \/private\/<\/code> \u043d\u0435 \u0437\u0430\u043a\u0440\u044b\u0442, \u0430 \u0441\u0435\u0440\u0432\u0435\u0440 \u043f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u044f\u0435\u0442 \u0441 \u043d\u0435\u0433\u043e \u043d\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\/private\/<\/code>. \u0421\u043a\u0440\u0438\u043f\u0442 \u043d\u0430 \u0442\u0430\u043a\u043e\u043c \u043e\u0442\u0432\u0435\u0442\u0435 \u043e\u0441\u0442\u0430\u043d\u0430\u0432\u043b\u0438\u0432\u0430\u0435\u0442\u0441\u044f \u0441 \u0441\u043e\u043e\u0431\u0449\u0435\u043d\u0438\u0435\u043c \u00ab\u0441\u0442\u043e\u043f, \u043f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435\u00bb.<\/p>\n<p><strong>\u0421\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">newline=\"\"<\/code> \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u043c\u043e\u0434\u0443\u043b\u044c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">csv<\/code>, \u0438\u043d\u0430\u0447\u0435 \u0432 Windows \u043f\u043e\u044f\u0432\u044f\u0442\u0441\u044f \u043f\u0443\u0441\u0442\u044b\u0435 \u0441\u0442\u0440\u043e\u043a\u0438. \u041a\u043e\u0434\u0438\u0440\u043e\u0432\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">utf-8-sig<\/code> \u0434\u043e\u0431\u0430\u0432\u043b\u044f\u0435\u0442 \u0432 \u043d\u0430\u0447\u0430\u043b\u043e \u0444\u0430\u0439\u043b\u0430 \u043c\u0435\u0442\u043a\u0443, \u043f\u043e \u043a\u043e\u0442\u043e\u0440\u043e\u0439 Excel \u0440\u0430\u0441\u043f\u043e\u0437\u043d\u0430\u0435\u0442 UTF-8.<\/p>\n<h2 id=\"s5\">\u0427\u0430\u0441\u0442\u0430\u044f \u043e\u0448\u0438\u0431\u043a\u0430: \u043a\u0440\u0430\u043a\u043e\u0437\u044f\u0431\u0440\u044b \u0432\u043c\u0435\u0441\u0442\u043e \u0440\u0443\u0441\u0441\u043a\u043e\u0433\u043e \u0442\u0435\u043a\u0441\u0442\u0430<\/h2>\n<p>\u041d\u0435\u0432\u0435\u0440\u043d\u044b\u0439 \u0432\u0430\u0440\u0438\u0430\u043d\u0442 &#8212; \u043e\u0442\u0434\u0430\u0442\u044c \u0432 BeautifulSoup <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">response.text<\/code>:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\"><span style=\"color: #C678DD\">import<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">requests<\/span>\n<span style=\"color: #C678DD\">from<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #E06C75\">bs4<\/span><span style=\"color: #ABB2BF\"> <\/span><span style=\"color: #C678DD\">import<\/span> <span style=\"color: #E06C75\">BeautifulSoup<\/span>\n\n<span style=\"color: #E06C75\">response<\/span> <span style=\"color: #56B6C2\">=<\/span> <span style=\"color: #E06C75\">requests<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;http:\/\/127.0.0.1:8000\/index.html&quot;<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #E06C75\">timeout<\/span><span style=\"color: #56B6C2\">=<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #D19A66\">5<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #D19A66\">15<\/span><span style=\"color: #ABB2BF\">))<\/span>\n<span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">headers<\/span><span style=\"color: #ABB2BF\">[<\/span><span style=\"color: #98C379\">&quot;Content-Type&quot;<\/span><span style=\"color: #ABB2BF\">],<\/span> <span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">encoding<\/span><span style=\"color: #ABB2BF\">)<\/span>\n<span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">BeautifulSoup<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">text<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;lxml&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select_one<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;h2 a&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get_text<\/span><span style=\"color: #ABB2BF\">())<\/span>\n<span style=\"color: #E5C07B\">print<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">BeautifulSoup<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #E06C75\">response<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">content<\/span><span style=\"color: #ABB2BF\">,<\/span> <span style=\"color: #98C379\">&quot;lxml&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">select_one<\/span><span style=\"color: #ABB2BF\">(<\/span><span style=\"color: #98C379\">&quot;h2 a&quot;<\/span><span style=\"color: #ABB2BF\">)<\/span><span style=\"color: #56B6C2\">.<\/span><span style=\"color: #E06C75\">get_text<\/span><span style=\"color: #ABB2BF\">())<\/span><\/code><\/pre>\n<p>\u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442 \u043d\u0430 \u0443\u0447\u0435\u0431\u043d\u043e\u043c \u0441\u0442\u0435\u043d\u0434\u0435:<\/p>\n<pre style=\"background:#282c34;color:#abb2bf;border-radius:8px;padding:16px 20px;margin:20px 0;overflow-x:auto;font-size:14px;line-height:1.55;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\"><code style=\"background:none\">text\/html ISO-8859-1\n\u00d0\u00d0\u00b7\u00d1\u00d1\u00d0\u00b0\u00d0\u00b5\u00d0\u00bc Python\n\u0418\u0437\u0443\u0447\u0430\u0435\u043c Python\n<\/code><\/pre>\n<p>\u0423\u0447\u0435\u0431\u043d\u044b\u0439 \u0441\u0435\u0440\u0432\u0435\u0440 \u043d\u0435 \u0443\u043a\u0430\u0437\u0430\u043b \u043a\u043e\u0434\u0438\u0440\u043e\u0432\u043a\u0443 \u0432 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0435 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Content-Type<\/code>, \u0438 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests<\/code> \u0434\u043b\u044f \u0442\u0435\u043a\u0441\u0442\u043e\u0432\u043e\u0433\u043e \u043e\u0442\u0432\u0435\u0442\u0430 \u043f\u043e\u0434\u0441\u0442\u0430\u0432\u0438\u043b ISO-8859-1. \u0418\u0441\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 &#8212; \u0442\u0440\u0435\u0442\u044c\u044f \u0441\u0442\u0440\u043e\u043a\u0430 \u0432\u044b\u0432\u043e\u0434\u0430: \u043f\u0435\u0440\u0435\u0434\u0430\u0432\u0430\u0442\u044c \u0431\u0430\u0439\u0442\u044b <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">response.content<\/code>, \u0442\u043e\u0433\u0434\u0430 BeautifulSoup \u043e\u043f\u0440\u0435\u0434\u0435\u043b\u044f\u0435\u0442 \u043a\u043e\u0434\u0438\u0440\u043e\u0432\u043a\u0443 \u0441\u0430\u043c, \u0432 \u0442\u043e\u043c \u0447\u0438\u0441\u043b\u0435 \u043f\u043e \u0442\u0435\u0433\u0443 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">&lt;meta charset&gt;<\/code>. \u0422\u0430\u043a \u0441\u0434\u0435\u043b\u0430\u043d\u043e \u0432 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">parser.py<\/code>.<\/p>\n<h2 id=\"s6\">\u0412\u0435\u0436\u043b\u0438\u0432\u043e\u0441\u0442\u044c \u0438 \u0431\u0435\u0437\u043e\u043f\u0430\u0441\u043d\u043e\u0441\u0442\u044c<\/h2>\n<ul>\n<li><strong><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>.<\/strong> \u0421\u043a\u0440\u0438\u043f\u0442 \u0437\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u0444\u0430\u0439\u043b \u043f\u0440\u0430\u0432\u0438\u043b \u0441 \u0442\u0435\u043c \u0436\u0435 \u0442\u0430\u0439\u043c\u0430\u0443\u0442\u043e\u043c \u0438 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">User-Agent<\/code>, \u0447\u0442\u043e \u0438 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b. \u041d\u0435\u0442 \u0444\u0430\u0439\u043b\u0430 (404) &#8212; \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d\u0438\u0439 \u043d\u0435\u0442; \u0441\u0435\u0440\u0432\u0435\u0440 \u043d\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d \u0438\u043b\u0438 \u043e\u0442\u0432\u0435\u0442\u0438\u043b \u0434\u0440\u0443\u0433\u043e\u0439 \u043e\u0448\u0438\u0431\u043a\u043e\u0439 &#8212; \u0440\u0430\u0431\u043e\u0442\u0430 \u043f\u0440\u0435\u043a\u0440\u0430\u0449\u0430\u0435\u0442\u0441\u044f, \u0430 \u043d\u0435 \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0430\u0435\u0442\u0441\u044f \u0432\u0441\u043b\u0435\u043f\u0443\u044e.<\/li>\n<li><strong>\u041f\u0430\u0443\u0437\u044b.<\/strong> \u041c\u0435\u0436\u0434\u0443 \u0437\u0430\u043f\u0440\u043e\u0441\u0430\u043c\u0438 \u0432\u044b\u0434\u0435\u0440\u0436\u0438\u0432\u0430\u0435\u0442\u0441\u044f <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Crawl-delay<\/code> \u0438\u0437 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>, \u0430 \u0435\u0441\u043b\u0438 \u0435\u0433\u043e \u043d\u0435\u0442 &#8212; \u043e\u0434\u043d\u0430 \u0441\u0435\u043a\u0443\u043d\u0434\u0430. \u0414\u0438\u0440\u0435\u043a\u0442\u0438\u0432\u0430 \u043d\u0435\u0441\u0442\u0430\u043d\u0434\u0430\u0440\u0442\u043d\u0430\u044f, \u0435\u0435 \u0443\u043a\u0430\u0437\u044b\u0432\u0430\u044e\u0442 \u043d\u0435 \u0432\u0441\u0435 \u0441\u0430\u0439\u0442\u044b; \u043f\u0430\u0443\u0437\u0430 \u0432 \u0441\u0435\u043a\u0443\u043d\u0434\u0443 &#8212; \u0440\u0430\u0437\u0443\u043c\u043d\u044b\u0439 \u043c\u0438\u043d\u0438\u043c\u0443\u043c \u0434\u043b\u044f \u043d\u0435\u0431\u043e\u043b\u044c\u0448\u043e\u0433\u043e \u0441\u0430\u0439\u0442\u0430, \u0430 \u043d\u0435 \u043d\u043e\u0440\u043c\u0430 \u043d\u0430 \u0432\u0441\u0435 \u0441\u043b\u0443\u0447\u0430\u0438.<\/li>\n<li><strong>\u0424\u043e\u0440\u043c\u0443\u043b\u044b \u0432 CSV.<\/strong> \u041d\u0430\u0437\u0432\u0430\u043d\u0438\u0435 \u043a\u043d\u0438\u0433\u0438 &#8212; \u0447\u0443\u0436\u0438\u0435 \u0434\u0430\u043d\u043d\u044b\u0435. \u042f\u0447\u0435\u0439\u043a\u0443, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043d\u0430\u0447\u0438\u043d\u0430\u0435\u0442\u0441\u044f \u0441 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">=<\/code>, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">+<\/code>, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">-<\/code> \u0438\u043b\u0438 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">@<\/code>, \u0442\u0430\u0431\u043b\u0438\u0447\u043d\u044b\u0439 \u0440\u0435\u0434\u0430\u043a\u0442\u043e\u0440 \u043c\u043e\u0436\u0435\u0442 \u0438\u0441\u043f\u043e\u043b\u043d\u0438\u0442\u044c \u043a\u0430\u043a \u0444\u043e\u0440\u043c\u0443\u043b\u0443. <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">safe_cell<\/code> \u0441\u0442\u0430\u0432\u0438\u0442 \u043f\u0435\u0440\u0435\u0434 \u0442\u0430\u043a\u0438\u043c \u0437\u043d\u0430\u0447\u0435\u043d\u0438\u0435\u043c \u0430\u043f\u043e\u0441\u0442\u0440\u043e\u0444; \u0441\u043f\u0438\u0441\u043e\u043a \u043f\u0435\u0440\u0432\u044b\u0445 \u0441\u0438\u043c\u0432\u043e\u043b\u043e\u0432 \u0432\u0437\u044f\u0442 \u0438\u0437 \u0440\u0435\u043a\u043e\u043c\u0435\u043d\u0434\u0430\u0446\u0438\u0438 OWASP \u043f\u043e CSV Injection \u0438 \u0432\u043a\u043b\u044e\u0447\u0430\u0435\u0442 \u0442\u0430\u043a\u0436\u0435 \u0442\u0430\u0431\u0443\u043b\u044f\u0446\u0438\u044e, \u0432\u043e\u0437\u0432\u0440\u0430\u0442 \u043a\u0430\u0440\u0435\u0442\u043a\u0438, \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0440\u043e\u043a\u0438 \u0438 \u043f\u043e\u043b\u043d\u043e\u0448\u0438\u0440\u0438\u043d\u043d\u044b\u0435 \u0432\u0430\u0440\u0438\u0430\u043d\u0442\u044b \u0437\u043d\u0430\u043a\u043e\u0432 (<code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\uff1d<\/code>, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\uff0b<\/code>, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\uff0d<\/code>, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\uff20<\/code>). \u042d\u0442\u043e \u0431\u0430\u0437\u043e\u0432\u0430\u044f \u043c\u0435\u0440\u0430 \u0434\u043b\u044f \u043f\u0435\u0440\u0432\u043e\u0433\u043e \u043e\u0442\u043a\u0440\u044b\u0442\u0438\u044f \u0444\u0430\u0439\u043b\u0430, \u0438 \u0435\u0435 \u0433\u0440\u0430\u043d\u0438\u0446\u0430 \u0437\u0430\u0432\u0438\u0441\u0438\u0442 \u043e\u0442 \u043f\u0440\u0438\u043b\u043e\u0436\u0435\u043d\u0438\u044f: \u043f\u043e \u043d\u0430\u0431\u043b\u044e\u0434\u0435\u043d\u0438\u044e OWASP, \u043f\u043e\u0441\u043b\u0435 \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0438 \u043f\u043e\u0432\u0442\u043e\u0440\u043d\u043e\u0433\u043e \u043e\u0442\u043a\u0440\u044b\u0442\u0438\u044f \u0432 Excel \u0437\u0430\u0449\u0438\u0442\u043d\u044b\u0439 \u0430\u043f\u043e\u0441\u0442\u0440\u043e\u0444 \u043c\u043e\u0436\u0435\u0442 \u0438\u0441\u0447\u0435\u0437\u043d\u0443\u0442\u044c. \u041f\u0440\u043e\u0432\u0435\u0440\u044f\u0439\u0442\u0435 \u043f\u043e\u043b\u043d\u044b\u0439 \u0446\u0438\u043a\u043b \u043e\u0442\u043a\u0440\u044b\u0442\u0438\u044f \u0438 \u044d\u043a\u0441\u043f\u043e\u0440\u0442\u0430 \u0432 \u0441\u0432\u043e\u0435\u043c \u0440\u0435\u0434\u0430\u043a\u0442\u043e\u0440\u0435 \u0438\u043b\u0438 \u0438\u043c\u043f\u043e\u0440\u0442\u0438\u0440\u0443\u0439\u0442\u0435 \u0447\u0443\u0436\u0438\u0435 \u043f\u043e\u043b\u044f \u043a\u0430\u043a \u0442\u0435\u043a\u0441\u0442. \u0417\u043d\u0430\u0447\u0435\u043d\u0438\u0435 \u0441 \u043f\u0440\u043e\u0431\u0435\u043b\u043e\u043c \u043f\u0435\u0440\u0435\u0434 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">=<\/code> \u0444\u0443\u043d\u043a\u0446\u0438\u044f \u043d\u0435 \u0442\u0440\u043e\u0433\u0430\u0435\u0442.<\/li>\n<li><strong>\u041f\u0440\u0430\u0432\u043e.<\/strong> <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code> &#8212; \u043f\u0440\u043e\u0441\u044c\u0431\u0430 \u0432\u043b\u0430\u0434\u0435\u043b\u044c\u0446\u0430, \u0430 \u043d\u0435 \u0442\u0435\u0445\u043d\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u0437\u0430\u0449\u0438\u0442\u0430 \u0438 \u043d\u0435 \u044e\u0440\u0438\u0434\u0438\u0447\u0435\u0441\u043a\u043e\u0435 \u0440\u0430\u0437\u0440\u0435\u0448\u0435\u043d\u0438\u0435. \u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c\u0441\u043a\u043e\u0435 \u0441\u043e\u0433\u043b\u0430\u0448\u0435\u043d\u0438\u0435 \u0441\u0430\u0439\u0442\u0430, \u0430\u0432\u0442\u043e\u0440\u0441\u043a\u0438\u0435 \u043f\u0440\u0430\u0432\u0430 \u0438 \u0437\u0430\u043a\u043e\u043d\u044b \u043e \u043f\u0435\u0440\u0441\u043e\u043d\u0430\u043b\u044c\u043d\u044b\u0445 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u0435\u0439\u0441\u0442\u0432\u0443\u044e\u0442 \u043d\u0435\u0437\u0430\u0432\u0438\u0441\u0438\u043c\u043e \u043e\u0442 \u043d\u0435\u0433\u043e.<\/li>\n<\/ul>\n<table>\n<thead>\n<tr>\n<th>\u0412 \u043f\u0440\u0438\u043c\u0435\u0440\u0435<\/th>\n<th>\u0412 \u0440\u0430\u0431\u043e\u0447\u0435\u043c \u043f\u0430\u0440\u0441\u0435\u0440\u0435 \u0434\u043e\u043f\u043e\u043b\u043d\u0438\u0442\u0435\u043b\u044c\u043d\u043e<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>\u043f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u043e\u0441\u0442\u0430\u043d\u0430\u0432\u043b\u0438\u0432\u0430\u0435\u0442 \u043e\u0431\u0445\u043e\u0434<\/td>\n<td>\u043f\u0435\u0440\u0435\u0445\u043e\u0434 \u043f\u043e \u0430\u0434\u0440\u0435\u0441\u0443 \u0438\u0437 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Location<\/code> \u043f\u043e\u0441\u043b\u0435 \u0442\u0435\u0445 \u0436\u0435 \u043f\u0440\u043e\u0432\u0435\u0440\u043e\u043a \u0445\u043e\u0441\u0442\u0430 \u0438 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>, \u0441 \u043b\u0438\u043c\u0438\u0442\u043e\u043c \u0447\u0438\u0441\u043b\u0430 \u043f\u0435\u0440\u0435\u0445\u043e\u0434\u043e\u0432<\/td>\n<\/tr>\n<tr>\n<td>\u043e\u0434\u0438\u043d \u043f\u043e\u0442\u043e\u043a, \u0444\u0438\u043a\u0441\u0438\u0440\u043e\u0432\u0430\u043d\u043d\u0430\u044f \u043f\u0430\u0443\u0437\u0430<\/td>\n<td>\u043f\u043e\u0432\u0442\u043e\u0440\u044b \u0441 \u0440\u0430\u0441\u0442\u0443\u0449\u0435\u0439 \u043f\u0430\u0443\u0437\u043e\u0439 \u043d\u0430 429 \u0438 5xx, \u0443\u0447\u0435\u0442 \u0437\u0430\u0433\u043e\u043b\u043e\u0432\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Retry-After<\/code><\/td>\n<\/tr>\n<tr>\n<td>\u0441\u043e\u043e\u0431\u0449\u0435\u043d\u0438\u044f \u0447\u0435\u0440\u0435\u0437 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">print<\/code><\/td>\n<td>\u0436\u0443\u0440\u043d\u0430\u043b \u0441 \u0430\u0434\u0440\u0435\u0441\u0430\u043c\u0438 \u0438 \u043a\u043e\u0434\u0430\u043c\u0438 \u043e\u0442\u0432\u0435\u0442\u043e\u0432<\/td>\n<\/tr>\n<tr>\n<td>\u0432\u0435\u0441\u044c \u0440\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442 \u0432 \u043f\u0430\u043c\u044f\u0442\u0438, \u0437\u0430\u043f\u0438\u0441\u044c \u0432 \u043a\u043e\u043d\u0446\u0435<\/td>\n<td>\u0437\u0430\u043f\u0438\u0441\u044c \u043f\u043e \u043c\u0435\u0440\u0435 \u043e\u0431\u0445\u043e\u0434\u0430, \u043f\u0440\u043e\u0434\u043e\u043b\u0436\u0435\u043d\u0438\u0435 \u0441 \u043c\u0435\u0441\u0442\u0430 \u043e\u0441\u0442\u0430\u043d\u043e\u0432\u043a\u0438<\/td>\n<\/tr>\n<tr>\n<td>\u0440\u0430\u0437\u043c\u0435\u0440 \u043e\u0442\u0432\u0435\u0442\u0430 \u043d\u0435 \u043e\u0433\u0440\u0430\u043d\u0438\u0447\u0435\u043d<\/td>\n<td>\u043b\u0438\u043c\u0438\u0442 \u0440\u0430\u0437\u043c\u0435\u0440\u0430 \u0438 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">Content-Type<\/code><\/td>\n<\/tr>\n<tr>\n<td>\u043f\u0443\u0441\u0442\u0430\u044f \u0446\u0435\u043d\u0430 \u043c\u043e\u043b\u0447\u0430<\/td>\n<td>\u043a\u043e\u043d\u0442\u0440\u043e\u043b\u044c \u0434\u043e\u043b\u0438 \u043f\u0443\u0441\u0442\u044b\u0445 \u043f\u043e\u043b\u0435\u0439: \u043f\u0440\u0438\u0437\u043d\u0430\u043a \u0441\u043c\u0435\u043d\u044b \u0440\u0430\u0437\u043c\u0435\u0442\u043a\u0438<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2 id=\"s7\">\u0415\u0441\u043b\u0438 \u043d\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u043b\u043e\u0441\u044c<\/h2>\n<table>\n<thead>\n<tr>\n<th>\u0421\u0438\u043c\u043f\u0442\u043e\u043c<\/th>\n<th>\u041f\u0440\u0438\u0447\u0438\u043d\u0430<\/th>\n<th>\u0427\u0442\u043e \u0434\u0435\u043b\u0430\u0442\u044c<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt \u043d\u0435\u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d: ConnectionError<\/code><\/td>\n<td>\u043d\u0435 \u0437\u0430\u043f\u0443\u0449\u0435\u043d \u0443\u0447\u0435\u0431\u043d\u044b\u0439 \u0441\u0435\u0440\u0432\u0435\u0440 \u0438\u043b\u0438 \u0437\u0430\u043d\u044f\u0442 \u043f\u043e\u0440\u0442 8000<\/td>\n<td>\u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">http.server<\/code> \u0432\u043e \u0432\u0442\u043e\u0440\u043e\u043c \u043e\u043a\u043d\u0435, \u0441\u0432\u0435\u0440\u0438\u0442\u044c \u043f\u043e\u0440\u0442 \u0432 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">START_URL<\/code><\/td>\n<\/tr>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">bs4.exceptions.FeatureNotFound<\/code><\/td>\n<td>\u043d\u0435 \u0443\u0441\u0442\u0430\u043d\u043e\u0432\u043b\u0435\u043d <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">lxml<\/code><\/td>\n<td>\u0443\u0441\u0442\u0430\u043d\u043e\u0432\u0438\u0442\u044c \u043f\u0430\u043a\u0435\u0442 \u0432 \u0442\u043e \u0436\u0435 \u043e\u043a\u0440\u0443\u0436\u0435\u043d\u0438\u0435, \u0438\u0437 \u043a\u043e\u0442\u043e\u0440\u043e\u0433\u043e \u0437\u0430\u043f\u0443\u0441\u043a\u0430\u0435\u0442\u0441\u044f \u0441\u043a\u0440\u0438\u043f\u0442<\/td>\n<\/tr>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\u043a\u0430\u0440\u0442\u043e\u0447\u0435\u043a 0<\/code> \u043d\u0430 \u0440\u0435\u0430\u043b\u044c\u043d\u043e\u043c \u0441\u0430\u0439\u0442\u0435<\/td>\n<td>\u0441\u0435\u043b\u0435\u043a\u0442\u043e\u0440 \u043d\u0435 \u043f\u043e\u0434\u0445\u043e\u0434\u0438\u0442 \u0438\u043b\u0438 \u0434\u0430\u043d\u043d\u044b\u0435 \u043f\u043e\u0434\u0433\u0440\u0443\u0436\u0430\u0435\u0442 JavaScript<\/td>\n<td>\u0441\u043e\u0445\u0440\u0430\u043d\u0438\u0442\u044c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">response.text<\/code> \u0432 \u0444\u0430\u0439\u043b \u0438 \u0438\u0441\u043a\u0430\u0442\u044c \u043d\u0443\u0436\u043d\u044b\u0439 \u0442\u0435\u043a\u0441\u0442 \u0432 \u043d\u0435\u043c, \u0430 \u043d\u0435 \u0432 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u0430\u0445 \u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a\u0430<\/td>\n<\/tr>\n<tr>\n<td><code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">\u043d\u0435 \u0437\u0430\u0433\u0440\u0443\u0436\u0435\u043d\u043e ...: HTTPError<\/code><\/td>\n<td>\u0441\u0430\u0439\u0442 \u043e\u0442\u0432\u0435\u0442\u0438\u043b 403, 404 \u0438\u043b\u0438 429<\/td>\n<td>\u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0442\u044c \u0430\u0434\u0440\u0435\u0441; \u043f\u0440\u0438 403 \u0438 429 \u0443\u0432\u0435\u043b\u0438\u0447\u0438\u0442\u044c \u043f\u0430\u0443\u0437\u044b, \u0430 \u043d\u0435 \u043c\u0430\u0441\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c\u0441\u044f \u043f\u043e\u0434 \u0431\u0440\u0430\u0443\u0437\u0435\u0440<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<h2 id=\"s8\">\u0412\u044b\u0432\u043e\u0434\u044b<\/h2>\n<ul>\n<li>\u041f\u0430\u0440\u0441\u0435\u0440 \u0441\u0430\u0439\u0442\u0430 &#8212; \u044d\u0442\u043e \u0447\u0435\u0442\u044b\u0440\u0435 \u0448\u0430\u0433\u0430: \u0437\u0430\u0433\u0440\u0443\u0437\u043a\u0430, \u0440\u0430\u0437\u0431\u043e\u0440 HTML, \u043e\u0447\u0438\u0441\u0442\u043a\u0430, \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435; \u043f\u0435\u0440\u0435\u0434 \u043d\u0438\u043c\u0438 \u0441\u0442\u043e\u0438\u0442 \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">robots.txt<\/code>.<\/li>\n<li>\u0421\u0432\u044f\u0437\u043a\u0430 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">requests<\/code> + <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">BeautifulSoup<\/code> + <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">lxml<\/code> \u0437\u0430\u043a\u0440\u044b\u0432\u0430\u0435\u0442 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0433\u0434\u0435 \u0434\u0430\u043d\u043d\u044b\u0435 \u0435\u0441\u0442\u044c \u043f\u0440\u044f\u043c\u043e \u0432 HTML-\u043e\u0442\u0432\u0435\u0442\u0435.<\/li>\n<li>\u041f\u043e\u043b\u044f \u043e\u0434\u043d\u043e\u0439 \u0437\u0430\u043f\u0438\u0441\u0438 \u0438\u0449\u0443\u0442 \u0432\u043d\u0443\u0442\u0440\u0438 \u0435\u0435 \u043a\u0430\u0440\u0442\u043e\u0447\u043a\u0438, \u0430 \u043e\u0442\u0441\u0443\u0442\u0441\u0442\u0432\u0438\u0435 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u0430 (<code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">None<\/code>) \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u044e\u0442 \u044f\u0432\u043d\u043e.<\/li>\n<li>\u0412 BeautifulSoup \u043f\u0435\u0440\u0435\u0434\u0430\u044e\u0442 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">response.content<\/code>, \u0447\u0442\u043e\u0431\u044b \u043d\u0435 \u043f\u043e\u043b\u0443\u0447\u0438\u0442\u044c \u0438\u0441\u043f\u043e\u0440\u0447\u0435\u043d\u043d\u0443\u044e \u043a\u043e\u0434\u0438\u0440\u043e\u0432\u043a\u0443.<\/li>\n<li>\u0422\u0430\u0439\u043c\u0430\u0443\u0442, \u043f\u0430\u0443\u0437\u0430, \u043b\u0438\u043c\u0438\u0442 \u0441\u0442\u0440\u0430\u043d\u0438\u0446, \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0430 \u0445\u043e\u0441\u0442\u0430 \u0438 \u0437\u0430\u0449\u0438\u0442\u0430 \u044f\u0447\u0435\u0435\u043a CSV &#8212; \u043e\u0431\u044f\u0437\u0430\u0442\u0435\u043b\u044c\u043d\u0430\u044f \u0447\u0430\u0441\u0442\u044c \u0434\u0430\u0436\u0435 \u0443\u0447\u0435\u0431\u043d\u043e\u0433\u043e \u0441\u043a\u0440\u0438\u043f\u0442\u0430.<\/li>\n<\/ul>\n<h2 id=\"s9\">\u0413\u0434\u0435 \u043f\u0440\u0438\u043c\u0435\u043d\u044f\u0435\u0442\u0441\u044f \/ \u0441\u0432\u044f\u0437\u044c \u0441 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u043e\u0439<\/h2>\n<p>\u041f\u0430\u0440\u0441\u0435\u0440\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0442 \u0434\u043b\u044f \u043c\u043e\u043d\u0438\u0442\u043e\u0440\u0438\u043d\u0433\u0430 \u0446\u0435\u043d, \u0441\u0431\u043e\u0440\u0430 \u0432\u0430\u043a\u0430\u043d\u0441\u0438\u0439 \u0438 \u043e\u0431\u044a\u044f\u0432\u043b\u0435\u043d\u0438\u0439, \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u0438 \u0441\u043e\u0431\u0441\u0442\u0432\u0435\u043d\u043d\u044b\u0445 \u0441\u0430\u0439\u0442\u043e\u0432 \u043d\u0430 \u0431\u0438\u0442\u044b\u0435 \u0441\u0441\u044b\u043b\u043a\u0438, \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043a\u0438 \u043d\u0430\u0431\u043e\u0440\u043e\u0432 \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f \u0430\u043d\u0430\u043b\u0438\u0442\u0438\u043a\u0438. \u0412 \u0440\u0430\u0431\u043e\u0447\u0438\u0445 \u043f\u0440\u043e\u0435\u043a\u0442\u0430\u0445 \u0432\u043e\u043a\u0440\u0443\u0433 \u0442\u043e\u0433\u043e \u0436\u0435 \u044f\u0434\u0440\u0430 \u043f\u043e\u044f\u0432\u043b\u044f\u044e\u0442\u0441\u044f \u043e\u0447\u0435\u0440\u0435\u0434\u0438 \u0437\u0430\u0434\u0430\u0447, \u0430\u0441\u0438\u043d\u0445\u0440\u043e\u043d\u043d\u044b\u0435 \u0437\u0430\u043f\u0440\u043e\u0441\u044b, \u0442\u0435\u0441\u0442\u044b \u043d\u0430 \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u043d\u044b\u0445 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430\u0445 \u0438 \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0432 \u0431\u0430\u0437\u0435 \u0434\u0430\u043d\u043d\u044b\u0445.<\/p>\n<div class=\"oj-cta\" style=\"background:#fbf4e4;border-left:4px solid #c9962b;border-radius:8px;padding:16px 20px;margin:24px 0\">\n<p style=\"margin:0 0 8px\"><strong>\u041e\u0441\u0432\u043e\u0439\u0442\u0435 \u0442\u0435\u043c\u0443 \u043d\u0430 \u043f\u0440\u0430\u043a\u0442\u0438\u043a\u0435<\/strong><\/p>\n<p>\u042d\u0442\u0438 \u0442\u0435\u043c\u044b &#8212; \u0430\u0441\u0438\u043d\u0445\u0440\u043e\u043d\u043d\u043e\u0441\u0442\u044c, \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435, \u0440\u0430\u0431\u043e\u0442\u0430 \u0441 \u0441\u0435\u0442\u044c\u044e \u0438 \u0431\u0430\u0437\u0430\u043c\u0438 &#8212; \u0441\u0438\u0441\u0442\u0435\u043c\u043d\u043e \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u044e\u0442\u0441\u044f \u043d\u0430 \u043a\u0443\u0440\u0441\u0435 <a href=\"https:\/\/otus.ru\/lessons\/python-professional\/?int_article=parser-opisanie-i-primer-koda&amp;int_place=article&amp;int_variant=v1\">Python-\u0440\u0430\u0437\u0440\u0430\u0431\u043e\u0442\u0447\u0438\u043a. \u041f\u0440\u043e\u0434\u0432\u0438\u043d\u0443\u0442\u044b\u0439 \u0443\u0440\u043e\u0432\u0435\u043d\u044c<\/a>. \u041f\u043e\u0437\u043d\u0430\u043a\u043e\u043c\u0438\u0442\u044c\u0441\u044f \u0441 \u0444\u043e\u0440\u043c\u0430\u0442\u043e\u043c \u0437\u0430\u043d\u044f\u0442\u0438\u0439 \u043c\u043e\u0436\u043d\u043e \u043d\u0430 <a href=\"https:\/\/otus.ru\/events\/?int_article=parser-opisanie-i-primer-koda&amp;int_place=article&amp;int_variant=v1\">\u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0443\u0440\u043e\u043a\u0430\u0445 Otus<\/a>.<\/p>\n<\/div>\n<p>\u0421\u043c\u0435\u0436\u043d\u044b\u0435 \u0442\u0435\u043c\u044b: <a href=\"\/journal\/parsing-v-php-chto-nuzhno-znat-novichkam\/\">\u043f\u0430\u0440\u0441\u0438\u043d\u0433 \u0432 PHP \u0434\u043b\u044f \u043d\u043e\u0432\u0438\u0447\u043a\u043e\u0432<\/a>.<\/p>\n<h2 id=\"s10\">FAQ<\/h2>\n<p><strong>\u0427\u0435\u043c <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">select<\/code> \u043e\u0442\u043b\u0438\u0447\u0430\u0435\u0442\u0441\u044f \u043e\u0442 <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">find_all<\/code> \u0432 BeautifulSoup?<\/strong><br \/>\n<code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">select<\/code> \u043f\u0440\u0438\u043d\u0438\u043c\u0430\u0435\u0442 CSS-\u0441\u0435\u043b\u0435\u043a\u0442\u043e\u0440, <code style=\"background:#eef0f3;color:#24292f;border-radius:4px;padding:1px 5px;font-size:0.92em;font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace\">find_all<\/code> &#8212; \u0438\u043c\u044f \u0442\u0435\u0433\u0430 \u0438 \u0444\u0438\u043b\u044c\u0442\u0440\u044b \u043f\u043e \u0430\u0442\u0440\u0438\u0431\u0443\u0442\u0430\u043c; \u043e\u0431\u0430 \u0432\u043e\u0437\u0432\u0440\u0430\u0449\u0430\u044e\u0442 \u0441\u043f\u0438\u0441\u043e\u043a \u043d\u0430\u0439\u0434\u0435\u043d\u043d\u044b\u0445 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u043e\u0432. \u0420\u0435\u0437\u0443\u043b\u044c\u0442\u0430\u0442 \u043e\u0434\u0438\u043d\u0430\u043a\u043e\u0432\u044b\u0439, \u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0432 \u043e\u0434\u043d\u043e\u043c \u0441\u043a\u0440\u0438\u043f\u0442\u0435 \u0434\u043e\u0441\u0442\u0430\u0442\u043e\u0447\u043d\u043e \u043f\u0440\u0438\u0434\u0435\u0440\u0436\u0438\u0432\u0430\u0442\u044c\u0441\u044f \u043e\u0434\u043d\u043e\u0433\u043e \u0441\u043f\u043e\u0441\u043e\u0431\u0430.<\/p>\n<p><strong>\u0427\u0442\u043e \u0434\u0435\u043b\u0430\u0442\u044c, \u0435\u0441\u043b\u0438 \u0434\u0430\u043d\u043d\u044b\u0435 \u043d\u0430 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0435 \u043f\u043e\u044f\u0432\u043b\u044f\u044e\u0442\u0441\u044f \u0442\u043e\u043b\u044c\u043a\u043e \u043f\u043e\u0441\u043b\u0435 \u0437\u0430\u0433\u0440\u0443\u0437\u043a\u0438 JavaScript?<\/strong><br \/>\n\u0421\u043d\u0430\u0447\u0430\u043b\u0430 \u043f\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u0442\u044c \u043d\u0430 \u0432\u043a\u043b\u0430\u0434\u043a\u0435 \u00ab\u0421\u0435\u0442\u044c\u00bb \u0432 \u0431\u0440\u0430\u0443\u0437\u0435\u0440\u0435, \u043a\u0430\u043a\u0438\u043c \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u043c \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430 \u043f\u043e\u043b\u0443\u0447\u0430\u0435\u0442 \u0434\u0430\u043d\u043d\u044b\u0435: \u0447\u0430\u0441\u0442\u043e \u044d\u0442\u043e JSON, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u043f\u0440\u043e\u0449\u0435 \u0437\u0430\u043f\u0440\u043e\u0441\u0438\u0442\u044c \u043d\u0430\u043f\u0440\u044f\u043c\u0443\u044e. \u0415\u0441\u043b\u0438 \u0442\u0430\u043a \u043d\u0435 \u0432\u044b\u0445\u043e\u0434\u0438\u0442, \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u044e\u0442 \u0443\u043f\u0440\u0430\u0432\u043b\u044f\u0435\u043c\u044b\u0439 \u0431\u0440\u0430\u0443\u0437\u0435\u0440 &#8212; Playwright \u0438\u043b\u0438 Selenium; \u044d\u0442\u043e \u043c\u0435\u0434\u043b\u0435\u043d\u043d\u0435\u0435 \u0438 \u0442\u0440\u0435\u0431\u0443\u0435\u0442 \u0431\u043e\u043b\u044c\u0448\u0435 \u0440\u0435\u0441\u0443\u0440\u0441\u043e\u0432.<\/p>\n<p><strong>\u0415\u0441\u0442\u044c \u043b\u0438 \u0443 \u0441\u0430\u0439\u0442\u0430 \u0441\u043f\u043e\u0441\u043e\u0431 \u043e\u0442\u0434\u0430\u0442\u044c \u0434\u0430\u043d\u043d\u044b\u0435 \u0431\u0435\u0437 \u043f\u0430\u0440\u0441\u0438\u043d\u0433\u0430?<\/strong><br \/>\n\u0427\u0430\u0441\u0442\u043e \u0435\u0441\u0442\u044c: \u043e\u0444\u0438\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0439 API, \u0432\u044b\u0433\u0440\u0443\u0437\u043a\u0438, RSS \u0438\u043b\u0438 \u043a\u0430\u0440\u0442\u0430 \u0441\u0430\u0439\u0442\u0430. \u0422\u0430\u043a\u043e\u0439 \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a \u0443\u0441\u0442\u043e\u0439\u0447\u0438\u0432\u0435\u0435 \u043a \u0441\u043c\u0435\u043d\u0435 \u0432\u0435\u0440\u0441\u0442\u043a\u0438, \u0438 \u0435\u0433\u043e \u0443\u0441\u043b\u043e\u0432\u0438\u044f \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u043d\u0438\u044f \u043e\u0431\u044b\u0447\u043d\u043e \u043e\u043f\u0438\u0441\u0430\u043d\u044b \u044f\u0432\u043d\u043e.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u041f\u0430\u0440\u0441\u0435\u0440 &#8212; \u044d\u0442\u043e \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u0430, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u0440\u0430\u0437\u0431\u0438\u0440\u0430\u0435\u0442 \u0434\u0430\u043d\u043d\u044b\u0435 \u043f\u043e \u0438\u0437\u0432\u0435\u0441\u0442\u043d\u044b\u043c \u043f\u0440\u0430\u0432\u0438\u043b\u0430\u043c \u0438 \u0434\u043e\u0441\u0442\u0430\u0435\u0442 \u0438\u0437 \u043d\u0438\u0445 \u043d\u0443\u0436\u043d\u044b\u0435 \u0447\u0430\u0441\u0442\u0438; \u0432 \u043e\u0431\u0438\u0445\u043e\u0434\u0435 \u0442\u0430\u043a \u0447\u0430\u0449\u0435 \u0432\u0441\u0435\u0433\u043e \u043d\u0430\u0437\u044b\u0432\u0430\u044e\u0442 \u0441\u043a\u0440\u0438\u043f\u0442, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u0433\u0440\u0443\u0436\u0430\u0435\u0442 \u0432\u0435\u0431-\u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u043d\u0430\u0445\u043e\u0434\u0438\u0442 \u0432 HTML \u043d\u0443\u0436\u043d\u044b\u0435 \u044d\u043b\u0435\u043c\u0435\u043d\u0442\u044b \u0438 \u0441\u043e\u0445\u0440\u0430\u043d\u044f\u0435\u0442 \u0438\u0445 \u0432 \u0442\u0430\u0431\u043b\u0438\u0446\u0443. \u041d\u0438\u0436\u0435 &#8212; \u0440\u0430\u0431\u043e\u0447\u0438\u0439 \u043f\u0440\u0438\u043c\u0435\u0440 \u0442\u0430\u043a\u043e\u0433\u043e \u043f\u0430\u0440\u0441\u0435\u0440\u0430 \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML, \u0437\u0430\u043f\u0438\u0441\u044c \u0432 CSV, \u0443\u0432\u0430\u0436\u0435\u043d\u0438\u0435 \u043a robots.txt, \u043f\u0430\u0443\u0437\u044b [&hellip;]<\/p>\n","protected":false},"author":11,"featured_media":9862,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[507],"class_list":["post-9859","post","type-post","status-publish","format-standard","has-post-thumbnail","","category-polza","tag-beautiful-soup"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043f\u0430\u0440\u0441\u0435\u0440 \u0438 \u043a\u0430\u043a \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0435\u0433\u043e \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML \u0447\u0435\u0440\u0435\u0437 BeautifulSoup \u0438 lxml, \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0432 CSV, robots.txt, \u043f\u0430\u0443\u0437\u044b \u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043e\u0448\u0438\u0431\u043e\u043a.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"A. Pavlenko\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<script type=\"application\/ld+json\" class=\"aioseo-schema\">\n\t\t\t{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#article\",\"name\":\"\\u041f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u043d\\u0430 Python: \\u043f\\u0440\\u0438\\u043c\\u0435\\u0440 \\u043a\\u043e\\u0434\\u0430 \\u0441 BeautifulSoup \\u0438 CSV | Otus\",\"headline\":\"\\u041f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u043d\\u0430 Python: \\u043e\\u043f\\u0438\\u0441\\u0430\\u043d\\u0438\\u0435 \\u0438 \\u043f\\u0440\\u0438\\u043c\\u0435\\u0440 \\u043a\\u043e\\u0434\\u0430\",\"author\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/author\\\/a-pavlenko\\\/#author\"},\"publisher\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#organization\"},\"image\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/oj-1080x720-93-1.jpg\",\"width\":2245,\"height\":1587},\"datePublished\":\"2026-09-30T09:00:00+00:00\",\"dateModified\":\"2026-09-30T15:52:57+00:00\",\"inLanguage\":\"ru-RU\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#webpage\"},\"isPartOf\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#webpage\"},\"articleSection\":\"\\u041f\\u043e\\u043b\\u0435\\u0437\\u043d\\u043e\\u0435, Beautiful Soup\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#breadcrumblist\",\"itemListElement\":[{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#listItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/otus.ru\\\/journal\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/category\\\/polza\\\/#listItem\",\"name\":\"\\u041f\\u043e\\u043b\\u0435\\u0437\\u043d\\u043e\\u0435\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/category\\\/polza\\\/#listItem\",\"position\":2,\"name\":\"\\u041f\\u043e\\u043b\\u0435\\u0437\\u043d\\u043e\\u0435\",\"item\":\"https:\\\/\\\/otus.ru\\\/journal\\\/category\\\/polza\\\/\",\"nextItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#listItem\",\"name\":\"\\u041f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u043d\\u0430 Python: \\u043e\\u043f\\u0438\\u0441\\u0430\\u043d\\u0438\\u0435 \\u0438 \\u043f\\u0440\\u0438\\u043c\\u0435\\u0440 \\u043a\\u043e\\u0434\\u0430\"},\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#listItem\",\"name\":\"Home\"}},{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#listItem\",\"position\":3,\"name\":\"\\u041f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u043d\\u0430 Python: \\u043e\\u043f\\u0438\\u0441\\u0430\\u043d\\u0438\\u0435 \\u0438 \\u043f\\u0440\\u0438\\u043c\\u0435\\u0440 \\u043a\\u043e\\u0434\\u0430\",\"previousItem\":{\"@type\":\"ListItem\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/category\\\/polza\\\/#listItem\",\"name\":\"\\u041f\\u043e\\u043b\\u0435\\u0437\\u043d\\u043e\\u0435\"}}]},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#organization\",\"name\":\"\\u041e\\u0442\\u0443\\u0441 \\u043e\\u043d\\u043b\\u0430\\u0439\\u043d-\\u043e\\u0431\\u0440\\u0430\\u0437\\u043e\\u0432\\u0430\\u043d\\u0438\\u0435\",\"description\":\"\\u0411\\u043b\\u043e\\u0433 \\u043f\\u0440\\u043e IT\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/\",\"telephone\":\"+74999389202\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/wp-content\\\/uploads\\\/2026\\\/04\\\/OTUS-journal-logo-180x180-1.png\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#organizationLogo\",\"width\":180,\"height\":180},\"image\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#organizationLogo\"},\"sameAs\":[\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCetgtvy93o3i3CvyGXKFU3g\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/author\\\/a-pavlenko\\\/#author\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/author\\\/a-pavlenko\\\/\",\"name\":\"A. Pavlenko\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#authorImage\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/13488c7d5b657aaaf1808afd398f71e0225df07dd94eb7e5fcdfc45feff43cb5?s=96&d=mm&r=g\",\"width\":96,\"height\":96,\"caption\":\"A. Pavlenko\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#webpage\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/\",\"name\":\"\\u041f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u043d\\u0430 Python: \\u043f\\u0440\\u0438\\u043c\\u0435\\u0440 \\u043a\\u043e\\u0434\\u0430 \\u0441 BeautifulSoup \\u0438 CSV | Otus\",\"description\":\"\\u0427\\u0442\\u043e \\u0442\\u0430\\u043a\\u043e\\u0435 \\u043f\\u0430\\u0440\\u0441\\u0435\\u0440 \\u0438 \\u043a\\u0430\\u043a \\u043d\\u0430\\u043f\\u0438\\u0441\\u0430\\u0442\\u044c \\u0435\\u0433\\u043e \\u043d\\u0430 Python: \\u0437\\u0430\\u043f\\u0440\\u043e\\u0441 \\u0441\\u0442\\u0440\\u0430\\u043d\\u0438\\u0446\\u044b, \\u0440\\u0430\\u0437\\u0431\\u043e\\u0440 HTML \\u0447\\u0435\\u0440\\u0435\\u0437 BeautifulSoup \\u0438 lxml, \\u0441\\u043e\\u0445\\u0440\\u0430\\u043d\\u0435\\u043d\\u0438\\u0435 \\u0432 CSV, robots.txt, \\u043f\\u0430\\u0443\\u0437\\u044b \\u0438 \\u043e\\u0431\\u0440\\u0430\\u0431\\u043e\\u0442\\u043a\\u0430 \\u043e\\u0448\\u0438\\u0431\\u043e\\u043a.\",\"inLanguage\":\"ru-RU\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#website\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#breadcrumblist\"},\"author\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/author\\\/a-pavlenko\\\/#author\"},\"creator\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/author\\\/a-pavlenko\\\/#author\"},\"image\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/wp-content\\\/uploads\\\/2024\\\/05\\\/oj-1080x720-93-1.jpg\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#mainImage\",\"width\":2245,\"height\":1587},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/parser-opisanie-i-primer-koda\\\/#mainImage\"},\"datePublished\":\"2026-09-30T09:00:00+00:00\",\"dateModified\":\"2026-09-30T15:52:57+00:00\"},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#website\",\"url\":\"https:\\\/\\\/otus.ru\\\/journal\\\/\",\"name\":\"OTUS \\u0416\\u0443\\u0440\\u043d\\u0430\\u043b - \\u0411\\u043b\\u043e\\u0433 \\u043f\\u0440\\u043e IT\",\"alternateName\":\"OTUS \\u0416\\u0443\\u0440\\u043d\\u0430\\u043b\",\"description\":\"\\u0411\\u043b\\u043e\\u0433 \\u043f\\u0440\\u043e IT\",\"inLanguage\":\"ru-RU\",\"publisher\":{\"@id\":\"https:\\\/\\\/otus.ru\\\/journal\\\/#organization\"}}]}\n\t\t<\/script>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u0441 BeautifulSoup \u0438 CSV | Otus","description":"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043f\u0430\u0440\u0441\u0435\u0440 \u0438 \u043a\u0430\u043a \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0435\u0433\u043e \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML \u0447\u0435\u0440\u0435\u0437 BeautifulSoup \u0438 lxml, \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0432 CSV, robots.txt, \u043f\u0430\u0443\u0437\u044b \u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043e\u0448\u0438\u0431\u043e\u043a.","canonical_url":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#article","name":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u0441 BeautifulSoup \u0438 CSV | Otus","headline":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430","author":{"@id":"https:\/\/otus.ru\/journal\/author\/a-pavlenko\/#author"},"publisher":{"@id":"https:\/\/otus.ru\/journal\/#organization"},"image":{"@type":"ImageObject","url":"https:\/\/otus.ru\/journal\/wp-content\/uploads\/2024\/05\/oj-1080x720-93-1.jpg","width":2245,"height":1587},"datePublished":"2026-09-30T09:00:00+00:00","dateModified":"2026-09-30T15:52:57+00:00","inLanguage":"ru-RU","mainEntityOfPage":{"@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#webpage"},"isPartOf":{"@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#webpage"},"articleSection":"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435, Beautiful Soup"},{"@type":"BreadcrumbList","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#breadcrumblist","itemListElement":[{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/#listItem","position":1,"name":"Home","item":"https:\/\/otus.ru\/journal\/","nextItem":{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/category\/polza\/#listItem","name":"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435"}},{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/category\/polza\/#listItem","position":2,"name":"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435","item":"https:\/\/otus.ru\/journal\/category\/polza\/","nextItem":{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#listItem","name":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430"},"previousItem":{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/#listItem","name":"Home"}},{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#listItem","position":3,"name":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430","previousItem":{"@type":"ListItem","@id":"https:\/\/otus.ru\/journal\/category\/polza\/#listItem","name":"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435"}}]},{"@type":"Organization","@id":"https:\/\/otus.ru\/journal\/#organization","name":"\u041e\u0442\u0443\u0441 \u043e\u043d\u043b\u0430\u0439\u043d-\u043e\u0431\u0440\u0430\u0437\u043e\u0432\u0430\u043d\u0438\u0435","description":"\u0411\u043b\u043e\u0433 \u043f\u0440\u043e IT","url":"https:\/\/otus.ru\/journal\/","telephone":"+74999389202","logo":{"@type":"ImageObject","url":"https:\/\/otus.ru\/journal\/wp-content\/uploads\/2026\/04\/OTUS-journal-logo-180x180-1.png","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#organizationLogo","width":180,"height":180},"image":{"@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#organizationLogo"},"sameAs":["https:\/\/www.youtube.com\/channel\/UCetgtvy93o3i3CvyGXKFU3g"]},{"@type":"Person","@id":"https:\/\/otus.ru\/journal\/author\/a-pavlenko\/#author","url":"https:\/\/otus.ru\/journal\/author\/a-pavlenko\/","name":"A. Pavlenko","image":{"@type":"ImageObject","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#authorImage","url":"https:\/\/secure.gravatar.com\/avatar\/13488c7d5b657aaaf1808afd398f71e0225df07dd94eb7e5fcdfc45feff43cb5?s=96&d=mm&r=g","width":96,"height":96,"caption":"A. Pavlenko"}},{"@type":"WebPage","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#webpage","url":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/","name":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u0441 BeautifulSoup \u0438 CSV | Otus","description":"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043f\u0430\u0440\u0441\u0435\u0440 \u0438 \u043a\u0430\u043a \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0435\u0433\u043e \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML \u0447\u0435\u0440\u0435\u0437 BeautifulSoup \u0438 lxml, \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0432 CSV, robots.txt, \u043f\u0430\u0443\u0437\u044b \u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043e\u0448\u0438\u0431\u043e\u043a.","inLanguage":"ru-RU","isPartOf":{"@id":"https:\/\/otus.ru\/journal\/#website"},"breadcrumb":{"@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#breadcrumblist"},"author":{"@id":"https:\/\/otus.ru\/journal\/author\/a-pavlenko\/#author"},"creator":{"@id":"https:\/\/otus.ru\/journal\/author\/a-pavlenko\/#author"},"image":{"@type":"ImageObject","url":"https:\/\/otus.ru\/journal\/wp-content\/uploads\/2024\/05\/oj-1080x720-93-1.jpg","@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#mainImage","width":2245,"height":1587},"primaryImageOfPage":{"@id":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/#mainImage"},"datePublished":"2026-09-30T09:00:00+00:00","dateModified":"2026-09-30T15:52:57+00:00"},{"@type":"WebSite","@id":"https:\/\/otus.ru\/journal\/#website","url":"https:\/\/otus.ru\/journal\/","name":"OTUS \u0416\u0443\u0440\u043d\u0430\u043b - \u0411\u043b\u043e\u0433 \u043f\u0440\u043e IT","alternateName":"OTUS \u0416\u0443\u0440\u043d\u0430\u043b","description":"\u0411\u043b\u043e\u0433 \u043f\u0440\u043e IT","inLanguage":"ru-RU","publisher":{"@id":"https:\/\/otus.ru\/journal\/#organization"}}]}},"aioseo_meta_data":{"post_id":"9859","title":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430 \u0441 BeautifulSoup \u0438 CSV | Otus","description":"\u0427\u0442\u043e \u0442\u0430\u043a\u043e\u0435 \u043f\u0430\u0440\u0441\u0435\u0440 \u0438 \u043a\u0430\u043a \u043d\u0430\u043f\u0438\u0441\u0430\u0442\u044c \u0435\u0433\u043e \u043d\u0430 Python: \u0437\u0430\u043f\u0440\u043e\u0441 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u044b, \u0440\u0430\u0437\u0431\u043e\u0440 HTML \u0447\u0435\u0440\u0435\u0437 BeautifulSoup \u0438 lxml, \u0441\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u0435 \u0432 CSV, robots.txt, \u043f\u0430\u0443\u0437\u044b \u0438 \u043e\u0431\u0440\u0430\u0431\u043e\u0442\u043a\u0430 \u043e\u0448\u0438\u0431\u043e\u043a.","keywords":[],"keyphrases":{"focus":{"keyphrase":"","score":0,"analysis":{"keyphraseInTitle":{"score":0,"maxScore":9,"error":1}}},"additional":[]},"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":"","og_custom_url":null,"og_article_section":null,"og_article_tags":[],"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"Article","isEnabled":true},"graphs":[]},"schema_type":"default","schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":"-1","robots_max_videopreview":"-1","robots_max_imagepreview":"large","priority":null,"frequency":"default","location":null,"local_seo":null,"breadcrumb_settings":null,"limit_modified_date":false,"ai":null,"created":"2024-05-13 10:53:33","updated":"2026-09-30 17:19:51","seo_analyzer_scan_date":null,"focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"aioseo_breadcrumb":"<div class=\"aioseo-breadcrumbs\"><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/otus.ru\/journal\/\" title=\"Home\">Home<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">\u00bb<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t<a href=\"https:\/\/otus.ru\/journal\/category\/polza\/\" title=\"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435\">\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435<\/a>\n\t\t<\/span><span class=\"aioseo-breadcrumb-separator\">\u00bb<\/span><span class=\"aioseo-breadcrumb\">\n\t\t\t\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430\n\t\t<\/span><\/div>","aioseo_breadcrumb_json":[{"label":"Home","link":"https:\/\/otus.ru\/journal\/"},{"label":"\u041f\u043e\u043b\u0435\u0437\u043d\u043e\u0435","link":"https:\/\/otus.ru\/journal\/category\/polza\/"},{"label":"\u041f\u0430\u0440\u0441\u0435\u0440 \u043d\u0430 Python: \u043e\u043f\u0438\u0441\u0430\u043d\u0438\u0435 \u0438 \u043f\u0440\u0438\u043c\u0435\u0440 \u043a\u043e\u0434\u0430","link":"https:\/\/otus.ru\/journal\/parser-opisanie-i-primer-koda\/"}],"_links":{"self":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/9859","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/comments?post=9859"}],"version-history":[{"count":5,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/9859\/revisions"}],"predecessor-version":[{"id":14105,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/posts\/9859\/revisions\/14105"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/media\/9862"}],"wp:attachment":[{"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/media?parent=9859"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/categories?post=9859"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/otus.ru\/journal\/wp-json\/wp\/v2\/tags?post=9859"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}