{"id":572,"date":"2023-04-18T16:27:41","date_gmt":"2023-04-18T14:27:41","guid":{"rendered":"https:\/\/jankiewicz.pl\/?p=572"},"modified":"2023-04-18T17:39:31","modified_gmt":"2023-04-18T15:39:31","slug":"mapreduce-w-trybie-local","status":"publish","type":"post","link":"https:\/\/jankiewicz.pl\/index.php\/mapreduce-w-trybie-local\/","title":{"rendered":"MapReduce \u2013 w trybie local"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Wprowadzenie<\/h2>\n\n\n\n<p>Nie zawsze program <em>MapReduce<\/em> jest na tyle prosty, \u017ce pierwsza pr\u00f3ba jego napisania ko\u0144czy si\u0119 sukcesem. <br>Co je\u015bli nasz program jest na tyle skomplikowany, \u017ce wymaga d\u0142ugotrwa\u0142ej implementacji? D\u0142ugotrwa\u0142e korzystanie z klastra mo\u017ce wi\u0105za\u0107 si\u0119 ze znacznymi kosztami. W takim przypadku by\u0142oby dobrze m\u00f3c uruchamia\u0107 nasz program <em>MapReduce<\/em> lokalnie.<\/p>\n\n\n\n<p>Du\u017ca cz\u0119\u015b\u0107 narz\u0119dzi Big Data posiada mo\u017cliwo\u015b\u0107 uruchamiania w tzw. trybie local. Tryb ten z regu\u0142y wykorzystuje w\u0105tki JVM analogicznie jak w przypadku klastra <em>Hadoop<\/em> wykorzystywane by\u0142yby kontenery YARN.<\/p>\n\n\n\n<p>Taki sam tryb ma <em>MapReduce<\/em>. W dawnych jego wersjach (<code>org.apache.hadoop.mapred.*<\/code>) uzyskiwa\u0142o si\u0119 go ustawiaj\u0105c po prostu parametr konfiguracyjny<\/p>\n\n\n\n<p><code>conf.set(\"mapreduce.framework.name\", \"local\");<\/code><\/p>\n\n\n\n<p>W przypadku nowszych bibliotek stosowane podej\u015bcie wygl\u0105da nieco inaczej. Gotowi?<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Punkt wyj\u015bcia<\/h2>\n\n\n\n<p>Zaczniemy od pocz\u0105tku.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Utw\u00f3rz projekt Javy w <em>IntelliJ IDE<\/em> o nazwie <code>WordCount<\/code><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-1.png\" alt=\"\" class=\"wp-image-573\" width=\"670\" height=\"283\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-1.png 992w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-1-300x127.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-1-768x325.png 768w\" sizes=\"auto, (max-width: 670px) 100vw, 670px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Utw\u00f3rz now\u0105 klas\u0119 Javy <code>WordCount<\/code><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-2.png\" alt=\"\" class=\"wp-image-574\" width=\"367\" height=\"146\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-2.png 629w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-2-300x119.png 300w\" sizes=\"auto, (max-width: 367px) 100vw, 367px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Wprowad\u017a do pliku tej klasy kod z dokumentacji <em>Apache<\/em> <em>Hadoop<\/em> <br><a href=\"https:\/\/hadoop.apache.org\/docs\/stable\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/MapReduceTutorial.html#Example:_WordCount_v1.0\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/hadoop.apache.org\/docs\/stable\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/MapReduceTutorial.html#Example:_WordCount_v1.0<\/a><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-3.png\" alt=\"\" class=\"wp-image-575\" width=\"429\" height=\"151\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-3.png 689w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-3-300x106.png 300w\" sizes=\"auto, (max-width: 429px) 100vw, 429px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Oczywi\u015bcie brakuje nam kilku bibliotek. Przejd\u017a do ustawie\u0144 projektu na zak\u0142adk\u0119 <em>Libraries<\/em><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-4.png\" alt=\"\" class=\"wp-image-576\" width=\"201\" height=\"196\"\/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-5.png\" alt=\"\" class=\"wp-image-577\" width=\"351\" height=\"172\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-5.png 489w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-5-300x147.png 300w\" sizes=\"auto, (max-width: 351px) 100vw, 351px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>A nast\u0119pnie dodaj nast\u0119puj\u0105ce biblioteki\n<ul class=\"wp-block-list\">\n<li><code>org.apache.hadoop:hadoop-common:3.3.5<\/code><\/li>\n\n\n\n<li><code>org.apache.hadoop:hadoop-mapreduce-client-common:3.3.5<\/code><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<p class=\"has-text-align-center\"><img loading=\"lazy\" decoding=\"async\" width=\"500\" height=\"189\" class=\"wp-image-578\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-6.png\" style=\"width: 500px;\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-6.png 533w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-6-300x114.png 300w\" sizes=\"auto, (max-width: 500px) 100vw, 500px\" \/> <\/p>\n\n\n\n<p>I tu do\u015b\u0107 istotna uwaga.<\/p>\n\n\n\n<p>Standardowo bibliotek\u0105 <em>MapReduce<\/em>, kt\u00f3r\u0105 potrzebujemy jest <br><code>org.apache.hadoop:hadoop-mapreduce-client-<strong>core<\/strong>:3.3.5<\/code> <\/p>\n\n\n\n<p>Jest ona w\u0142a\u015bciwym rozwi\u0105zaniem je\u015bli chcemy uruchamia\u0107 zadania <em>MapReduce<\/em> w klastrach <em>Hadoop,<\/em> gdzie jest dost\u0119pny zar\u00f3wno YARN jak i HDFS, <br>patrz: <a href=\"https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/dependency-analysis.html\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/dependency-analysis.html<\/a> <br><br>my skorzystali\u015bmy z biblioteki <br><code>org.apache.hadoop:hadoop-mapreduce-client-<strong>common<\/strong>:3.3.5<\/code><\/p>\n\n\n\n<p>kt\u00f3ra takich mo\u017cliwo\u015bci nie posiada, <br>patrz: <a rel=\"noreferrer noopener\" href=\"https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-common\/dependency-analysis.html\" target=\"_blank\">https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-common\/dependency-analysis.html<\/a> <br><br>Jednak w zamian za to posiada np. klas\u0119 <code>org.apache.hadoop.mapred.<strong>Local<\/strong>JobRunner<\/code> i dzi\u0119ki niej pozwala na bezproblemowe uruchamianie i testowanie zada\u0144 <em>MapReduce<\/em> w trybie <em>local<\/em>, jak b\u0119dziemy mieli si\u0119 okazj\u0119 za chwil\u0119 przekona\u0107.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Pobierz przyk\u0142adowe dane, kt\u00f3re b\u0119dziemy przetwarzali np.: <br><code><a rel=\"noreferrer noopener\" href=\"https:\/\/sherlock-holm.es\/stories\/plain-text\/cano.txt\" target=\"_blank\">https:\/\/sherlock-holm.es\/stories\/plain-text\/cano.txt<\/a><\/code> <br>i umie\u015b\u0107 je w katalogu np. <code>c:\\tmp\\input\\<\/code><\/li>\n\n\n\n<li>Uruchom nasz program po raz pierwszy. Efekt nie b\u0119dzie pozytywny<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-7.png\" alt=\"\" class=\"wp-image-579\" width=\"325\" height=\"120\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-7.png 467w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-7-300x111.png 300w\" sizes=\"auto, (max-width: 325px) 100vw, 325px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-8.png\" alt=\"\" class=\"wp-image-580\" width=\"604\" height=\"75\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-8.png 986w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-8-300x38.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-8-768x97.png 768w\" sizes=\"auto, (max-width: 604px) 100vw, 604px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Pow\u00f3d jest oczywisty. Nasz kod wymaga podania parametr\u00f3w<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-9.png\" alt=\"\" class=\"wp-image-581\" width=\"565\" height=\"101\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-9.png 807w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-9-300x54.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-9-768x137.png 768w\" sizes=\"auto, (max-width: 565px) 100vw, 565px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Wprowad\u017amy zatem zmiany w konfiguracji uruchamiania naszego programu. W za\u0142o\u017ceniu katalog <code>c:\\tmp\\output<\/code> nie istnieje \u2013 b\u0119dzie on utworzony przez zadanie <em>MapReduce<\/em><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-10.png\" alt=\"\" class=\"wp-image-582\" width=\"275\" height=\"85\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-10.png 395w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-10-300x93.png 300w\" sizes=\"auto, (max-width: 275px) 100vw, 275px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-11.png\" alt=\"\" class=\"wp-image-583\" width=\"272\" height=\"120\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-11.png 398w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-11-300x133.png 300w\" sizes=\"auto, (max-width: 272px) 100vw, 272px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Ponownie uruchom nasz\u0105 aplikacj\u0119.<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-12.png\" alt=\"\" class=\"wp-image-584\" width=\"384\" height=\"77\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-12.png 576w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-12-300x60.png 300w\" sizes=\"auto, (max-width: 384px) 100vw, 384px\" \/><\/figure>\n<\/div>\n\n\n<p>i\u2026 je\u015bli masz ustawion\u0105 zmienn\u0105 \u015brodowiskow\u0105 <code>HADOOP_HOME<\/code> wskazuj\u0105c\u0105 na odpowiednie katalog domowy oprogramowania Hadoop to\u2026 <br>dane z katalogu <code>c:\\tmp\\input<\/code> zosta\u0142y przetworzone, a wyniki umieszczone w katalogu <code>c:\\tmp\\ouput<\/code><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-13.png\" alt=\"\" class=\"wp-image-585\" width=\"325\" height=\"210\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-13.png 491w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-13-300x194.png 300w\" sizes=\"auto, (max-width: 325px) 100vw, 325px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-14.png\" alt=\"\" class=\"wp-image-586\" width=\"211\" height=\"140\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-14.png 317w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-14-300x200.png 300w\" sizes=\"auto, (max-width: 211px) 100vw, 211px\" \/><\/figure>\n<\/div>\n\n\n<p>Je\u015bli jednak te zmiennej <code>HADOOP_HOME<\/code> nie posiadasz to rezultat jest zapewne podobny do tego poni\u017cej<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-15.png\" alt=\"\" class=\"wp-image-587\" width=\"679\" height=\"85\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-15.png 1014w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-15-300x38.png 300w\" sizes=\"auto, (max-width: 679px) 100vw, 679px\" \/><\/figure>\n<\/div>\n\n\n<p>W takim przypadku potrzebny jest jeszcze jeden etap<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u015arodowisko<\/h2>\n\n\n\n<p>W przypadku \u015brodowiska Linux wystarczy zapewne<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>pobra\u0107 w\u0142a\u015bciw\u0105 wersj\u0119 oprogramowania <em>Apache Hadoop<\/em><a href=\"https:\/\/hadoop.apache.org\/releases.html\" target=\"_blank\" rel=\"noreferrer noopener\"> https:\/\/hadoop.apache.org\/releases.html<\/a><\/li>\n\n\n\n<li>rozpakowa\u0107 j\u0105 do wybranego katalogu, a nast\u0119pnie<\/li>\n\n\n\n<li>zdefiniowa\u0107 zmienn\u0105 HADOOP_HOME<\/li>\n<\/ul>\n\n\n\n<p>W przypadku \u015brodowiska Windows sytuacja jest nieco odmienna. <em>Apache Hadoop<\/em> nie jest przeznaczony dla \u015brodowiska Windows. Na szcz\u0119\u015bcie stworzone zosta\u0142o specjalne oprogramowanie <code>winutils<\/code>, kt\u00f3re ten problem rozwi\u0105zuje.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Pobierz odpowiedni\u0105 wersj\u0119 oprogramowania ze strony <a href=\"https:\/\/github.com\/cdarlint\/winutils\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/github.com\/cdarlint\/winutils<\/a><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-16.png\" alt=\"\" class=\"wp-image-588\" width=\"481\" height=\"196\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-16.png 765w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-16-300x122.png 300w\" sizes=\"auto, (max-width: 481px) 100vw, 481px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Nast\u0119pnie ustaw\/dodaj zmienn\u0105 \u015brodowiskow\u0105 <code>HADOOP_HOME<\/code>, tak aby wskazywa\u0142a na w\u0142a\u015bciwy katalog.<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-17.png\" alt=\"\" class=\"wp-image-589\" width=\"270\" height=\"85\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-17.png 458w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-17-300x94.png 300w\" sizes=\"auto, (max-width: 270px) 100vw, 270px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-18.png\" alt=\"\" class=\"wp-image-590\" width=\"202\" height=\"56\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-18.png 344w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-18-300x84.png 300w\" sizes=\"auto, (max-width: 202px) 100vw, 202px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-19.png\" alt=\"\" class=\"wp-image-591\" width=\"461\" height=\"82\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-19.png 707w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-19-300x53.png 300w\" sizes=\"auto, (max-width: 461px) 100vw, 461px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Zmie\u0144 te\u017c zmienn\u0105 \u015brodowiskow\u0105 <code>PATH<\/code>, tak aby zawiera\u0142a <code>%HADOOP_HOME%\\bin<\/code><\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-20.png\" alt=\"\" class=\"wp-image-592\" width=\"484\" height=\"85\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-20.png 700w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-20-300x53.png 300w\" sizes=\"auto, (max-width: 484px) 100vw, 484px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-21.png\" alt=\"\" class=\"wp-image-593\" width=\"249\" height=\"63\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-21.png 345w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-21-300x76.png 300w\" sizes=\"auto, (max-width: 249px) 100vw, 249px\" \/><\/figure>\n<\/div>\n\n\n<ul class=\"wp-block-list\">\n<li>Zrestartuj <em>IntelliJ IDEA<\/em>, aby zmienne \u015brodowiskowe zosta\u0142y ponownie zaczytane i\u2026 gotowe.<\/li>\n<\/ul>\n\n\n\n<p>Dane z katalogu <code>c:\\tmp\\input<\/code> zosta\u0142y przetworzone, a wyniki umieszczone w katalogu <code>c:\\tmp\\ouput<\/code><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-22.png\" alt=\"\" class=\"wp-image-594\" width=\"604\" height=\"82\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-22.png 885w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-22-300x41.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-22-768x105.png 768w\" sizes=\"auto, (max-width: 604px) 100vw, 604px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-23.png\" alt=\"\" class=\"wp-image-595\" width=\"343\" height=\"221\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-23.png 491w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-23-300x194.png 300w\" sizes=\"auto, (max-width: 343px) 100vw, 343px\" \/><\/figure>\n<\/div>\n\n\n<p>Uwaga! Domy\u015blnie silnik <em>MapReduce<\/em> wykorzystuje jeden reduktor. <br>Patrz: <a href=\"https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/mapred-default.xml\" target=\"_blank\" rel=\"noreferrer noopener\">https:\/\/hadoop.apache.org\/docs\/r3.3.5\/hadoop-mapreduce-client\/hadoop-mapreduce-client-core\/mapred-default.xml<\/a><\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-24.png\" alt=\"\" class=\"wp-image-596\" width=\"350\" height=\"44\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-24.png 570w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-24-300x38.png 300w\" sizes=\"auto, (max-width: 350px) 100vw, 350px\" \/><\/figure>\n<\/div>\n\n\n<p>Pliki konfiguracyjne mog\u0105 to oczywi\u015bcie zmieni\u0107.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-25.png\" alt=\"\" class=\"wp-image-597\" width=\"708\" height=\"214\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-25.png 1354w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-25-300x91.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-25-1024x311.png 1024w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-25-768x233.png 768w\" sizes=\"auto, (max-width: 708px) 100vw, 708px\" \/><\/figure>\n<\/div>\n\n\n<p>Jednak nale\u017cy pami\u0119ta\u0107, \u017ce<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-26.png\" alt=\"\" class=\"wp-image-598\" width=\"564\" height=\"67\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-26.png 951w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-26-300x36.png 300w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-26-768x93.png 768w\" sizes=\"auto, (max-width: 564px) 100vw, 564px\" \/><\/figure>\n<\/div>\n\n\n<p>Zatem pozostaje nam alternatywa polegaj\u0105ca na okre\u015bleniu liczby reduktor\u00f3w podczas konfiguracji zadania <em>MapReduce<\/em>.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-27.png\" alt=\"\" class=\"wp-image-599\" width=\"241\" height=\"53\" srcset=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-27.png 349w, https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-27-300x66.png 300w\" sizes=\"auto, (max-width: 241px) 100vw, 241px\" \/><\/figure>\n<\/div>\n\n\n<p>Zdecydowanie zalecam dokonanie korekty tego elementu, aby lepiej odwzorowa\u0107 rozproszone dzia\u0142ania, kt\u00f3re maj\u0105 miejsce w klastrze Hadoop.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/jankiewicz.pl\/wp-content\/uploads\/2023\/04\/word-image-572-28.png\" alt=\"\" class=\"wp-image-600\" width=\"189\" height=\"131\"\/><\/figure>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>Nie zawsze program MapReduce jest na tyle prosty, \u017ce pierwsza pr\u00f3ba jego napisania ko\u0144czy si\u0119 sukcesem.<br \/>\nCo je\u015bli nasz program jest na tyle skomplikowany, \u017ce wymaga d\u0142ugotrwa\u0142ej implementacji? D\u0142ugotrwa\u0142e korzystanie z klastra mo\u017ce wi\u0105za\u0107 si\u0119 ze znacznymi kosztami. W takim przypadku by\u0142oby dobrze m\u00f3c uruchamia\u0107 nasz program MapReduce lokalnie.<\/p>\n","protected":false},"author":2,"featured_media":611,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[55],"tags":[],"class_list":["post-572","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-stuku-puku"],"_links":{"self":[{"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/posts\/572","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/comments?post=572"}],"version-history":[{"count":4,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/posts\/572\/revisions"}],"predecessor-version":[{"id":620,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/posts\/572\/revisions\/620"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/media\/611"}],"wp:attachment":[{"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/media?parent=572"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/categories?post=572"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jankiewicz.pl\/index.php\/wp-json\/wp\/v2\/tags?post=572"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}