diff --git a/map-reduce/README.md b/map-reduce/README.md new file mode 100644 index 000000000000..2b0b150033bd --- /dev/null +++ b/map-reduce/README.md @@ -0,0 +1,231 @@ +--- +title: "MapReduce Pattern in Java" +shortTitle: MapReduce +description: "Learn the MapReduce pattern in Java with real-world examples, class diagrams, and tutorials. Understand its intent, applicability, benefits, and known uses to enhance your design pattern knowledge." +category: Performance optimization +language: en +tag: + - Data processing + - Code simplification + - Delegation + - Performance +--- + +## Also known as + +* Split-Apply-Combine Strategy +* Scatter-Gather Pattern + +## Intent of Map Reduce Design Pattern + +MapReduce aims to process and generate large datasets with a parallel, distributed algorithm on a cluster. It divides the workload into two main phases: Map and Reduce, allowing for efficient parallel processing of data. + +## Detailed Explanation of Map Reduce Pattern with Real-World Examples + +Real-world example + +> Imagine a large e-commerce company that wants to analyze its sales data across multiple regions. They have terabytes of transaction data stored across hundreds of servers. Using MapReduce, they can efficiently process this data to calculate total sales by product category. The Map function would process individual sales records, emitting key-value pairs of (category, sale amount). The Reduce function would then sum up all sale amounts for each category, producing the final result. + +In plain words + +> MapReduce splits a large problem into smaller parts, processes them in parallel, and then combines the results. + +Wikipedia says + +> "MapReduce is a programming model and associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster". +MapReduce consists of two main steps: +The "Map" step: The master node takes the input, divides it into smaller sub-problems, and distributes them to worker nodes. A worker node may do this again in turn, leading to a multi-level tree structure. The worker node processes the smaller problem, and passes the answer back to its master node. +The "Reduce" step: The master node then collects the answers to all the sub-problems and combines them in some way to form the output – the answer to the problem it was originally trying to solve. +This approach allows for efficient processing of vast amounts of data across multiple machines, making it a fundamental technique in big data analytics and distributed computing. + +## Programmatic Example of Map Reduce in Java + +### 1. Map Phase (Splitting & Processing Data) + +* The Mapper takes an input string, splits it into words, and counts occurrences. +* Output: A map {word → count} for each input line. +#### `Mapper.java` +```java +public class Mapper { + public static Map map(String input) { + Map wordCount = new HashMap<>(); + String[] words = input.split("\\s+"); + for (String word : words) { + word = word.toLowerCase().replaceAll("[^a-z]", ""); + if (!word.isEmpty()) { + wordCount.put(word, wordCount.getOrDefault(word, 0) + 1); + } + } + return wordCount; + } +} +``` +Example Input: ```"Hello world hello"``` +Output: ```{hello=2, world=1}``` + +### 2. Shuffle Phase (Grouping Data by Key) + +* The Shuffler collects key-value pairs from multiple mappers and groups values by key. +#### `Shuffler.java` +```java +public class Shuffler { + public static Map> shuffleAndSort(List> mapped) { + Map> grouped = new HashMap<>(); + for (Map map : mapped) { + for (Map.Entry entry : map.entrySet()) { + grouped.putIfAbsent(entry.getKey(), new ArrayList<>()); + grouped.get(entry.getKey()).add(entry.getValue()); + } + } + return grouped; + } +} +``` +Example Input: +``` +[ + {"hello": 2, "world": 1}, + {"hello": 1, "java": 1} +] +``` +Output: +``` +{ + "hello": [2, 1], + "world": [1], + "java": [1] +} +``` + +### 3. Reduce Phase (Aggregating Results) + +* The Reducer sums up occurrences of each word. +#### `Reducer.java` +```java +public class Reducer { + public static List> reduce(Map> grouped) { + Map reduced = new HashMap<>(); + for (Map.Entry> entry : grouped.entrySet()) { + reduced.put(entry.getKey(), entry.getValue().stream().mapToInt(Integer::intValue).sum()); + } + + List> result = new ArrayList<>(reduced.entrySet()); + result.sort(Map.Entry.comparingByValue(Comparator.reverseOrder())); + return result; + } +} +``` +Example Input: +``` +{ + "hello": [2, 1], + "world": [1], + "java": [1] +} +``` +Output: +``` +[ + {"hello": 3}, + {"world": 1}, + {"java": 1} +] +``` + +### 4. Running the Full MapReduce Process + +* The MapReduce class coordinates the three steps. +#### `MapReduce.java` +```java +public class MapReduce { + public static List> mapReduce(List inputs) { + List> mapped = new ArrayList<>(); + for (String input : inputs) { + mapped.add(Mapper.map(input)); + } + + Map> grouped = Shuffler.shuffleAndSort(mapped); + + return Reducer.reduce(grouped); + } +} +``` + +### 4. Main Execution (Calling MapReduce) + +* The Main class executes the MapReduce pipeline and prints the final word count. +#### `Main.java` +```java + public static void main(String[] args) { + List inputs = Arrays.asList( + "Hello world hello", + "MapReduce is fun", + "Hello from the other side", + "Hello world" + ); + List> result = MapReduce.mapReduce(inputs); + for (Map.Entry entry : result) { + System.out.println(entry.getKey() + ": " + entry.getValue()); + } +} +``` + +Output: +``` +hello: 4 +world: 2 +the: 1 +other: 1 +side: 1 +mapreduce: 1 +is: 1 +from: 1 +fun: 1 +``` + +## When to Use the Map Reduce Pattern in Java + +Use MapReduce when: +* Processing large datasets that don't fit into a single machine's memory +* Performing computations that can be parallelized +* Dealing with fault-tolerant and distributed computing scenarios +* Analyzing log files, web crawl data, or scientific data + +## Map Reduce Pattern Java Tutorials + +* [MapReduce Tutorial(Apache Hadoop)](https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) +* [MapReduce Example(Simplilearn)](https://www.youtube.com/watch?v=l2clwKnrtO8) + +## Benefits and Trade-offs of Map Reduce Pattern + +Benefits: + +* Scalability: Can process vast amounts of data across multiple machines +* Fault-tolerance: Handles machine failures gracefully +* Simplicity: Abstracts complex distributed computing details + +Trade-offs: + +* Overhead: Not efficient for small datasets due to setup and coordination costs +* Limited flexibility: Not suitable for all types of computations or algorithms +* Latency: Batch-oriented nature may not be suitable for real-time processing needs + +## Real-World Applications of Map Reduce Pattern in Java + +* Google's original implementation for indexing web pages +* Hadoop MapReduce for big data processing +* Log analysis in large-scale systems +* Genomic sequence analysis in bioinformatics + +## Related Java Design Patterns + +* Chaining Pattern +* Master-Worker Pattern +* Pipeline Pattern + +## References and Credits + +* [What is MapReduce](https://www.ibm.com/think/topics/mapreduce) +* [Wy MapReduce is not dead](https://www.codemotion.com/magazine/ai-ml/big-data/mapreduce-not-dead-heres-why-its-still-ruling-in-the-cloud/) +* [Scalabe Distributed Data Processing Solutions](https://tcpp.cs.gsu.edu/curriculum/?q=system%2Ffiles%2Fch07.pdf) +* [Java Design Patterns: A Hands-On Experience with Real-World Examples](https://amzn.to/3HWNf4U) diff --git a/map-reduce/etc/map-reduce.png b/map-reduce/etc/map-reduce.png new file mode 100644 index 000000000000..caf764553291 Binary files /dev/null and b/map-reduce/etc/map-reduce.png differ diff --git a/map-reduce/etc/map-reduce.urm.puml b/map-reduce/etc/map-reduce.urm.puml new file mode 100644 index 000000000000..508f1fd3363d --- /dev/null +++ b/map-reduce/etc/map-reduce.urm.puml @@ -0,0 +1,24 @@ +@startuml +package com.iluwatar { + class Main { + + Main() + + main(args : String[]) {static} + } + class MapReduce { + + MapReduce() + + mapReduce(inputs : List) : List> {static} + } + class Mapper { + + Mapper() + + map(input : String) : Map {static} + } + class Reducer { + + Reducer() + + reduce(grouped : Map>) : List> {static} + } + class Shuffler { + + Shuffler() + + shuffleAndSort(mapped : List>) : Map> {static} + } +} +@enduml \ No newline at end of file diff --git a/map-reduce/pom.xml b/map-reduce/pom.xml new file mode 100644 index 000000000000..4778f1bdb7ca --- /dev/null +++ b/map-reduce/pom.xml @@ -0,0 +1,62 @@ + + + + 4.0.0 + + com.iluwatar + java-design-patterns + 1.26.0-SNAPSHOT + + map-reduce + + + org.junit.jupiter + junit-jupiter-engine + test + + + + + + org.apache.maven.plugins + maven-assembly-plugin + + + + + + com.iluwatar.mapreduce.Main + + + + + + + + + diff --git a/map-reduce/src/main/java/com/iluwatar/Main.java b/map-reduce/src/main/java/com/iluwatar/Main.java new file mode 100644 index 000000000000..cdbc809ae381 --- /dev/null +++ b/map-reduce/src/main/java/com/iluwatar/Main.java @@ -0,0 +1,55 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import java.util.Arrays; +import java.util.List; +import java.util.Map; +import java.util.logging.Logger; + +/** + * The Main class serves as the entry point for executing the MapReduce program. + * It processes a list of text inputs, applies the MapReduce pattern, and prints the results. + */ +public class Main { + private static final Logger logger = Logger.getLogger(Main.class.getName()); + /** + * The main method initiates the MapReduce process and displays the word count results. + * + * @param args Command-line arguments (not used). + */ + public static void main(String[] args) { + List inputs = Arrays.asList( + "Hello world hello", + "MapReduce is fun", + "Hello from the other side", + "Hello world" + ); + List> result = MapReduce.mapReduce(inputs); + for (Map.Entry entry : result) { + logger.info(entry.getKey() + ": " + entry.getValue()); + } + } +} \ No newline at end of file diff --git a/map-reduce/src/main/java/com/iluwatar/MapReduce.java b/map-reduce/src/main/java/com/iluwatar/MapReduce.java new file mode 100644 index 000000000000..86964d5dbd48 --- /dev/null +++ b/map-reduce/src/main/java/com/iluwatar/MapReduce.java @@ -0,0 +1,55 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +/** + * The MapReduce class orchestrates the MapReduce process, + * calling the Mapper, Shuffler, and Reducer components. + */ +public class MapReduce { + private MapReduce() { + throw new UnsupportedOperationException("MapReduce is a utility class and cannot be instantiated."); + } + /** + * Executes the MapReduce process on the given list of input strings. + * + * @param inputs List of input strings to be processed. + * @return A list of word counts sorted in descending order. + */ + public static List> mapReduce(List inputs) { + List> mapped = new ArrayList<>(); + for (String input : inputs) { + mapped.add(Mapper.map(input)); + } + + Map> grouped = Shuffler.shuffleAndSort(mapped); + + return Reducer.reduce(grouped); + } +} diff --git a/map-reduce/src/main/java/com/iluwatar/Mapper.java b/map-reduce/src/main/java/com/iluwatar/Mapper.java new file mode 100644 index 000000000000..5a21a73f0d4a --- /dev/null +++ b/map-reduce/src/main/java/com/iluwatar/Mapper.java @@ -0,0 +1,56 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import java.util.HashMap; +import java.util.Map; + + +/** + * The Mapper class is responsible for processing an input string + * and generating a map of word occurrences. + */ +public class Mapper { + private Mapper() { + throw new UnsupportedOperationException("Mapper is a utility class and cannot be instantiated."); + } + /** + * Splits a given input string into words and counts their occurrences. + * + * @param input The input string to be mapped. + * @return A map where keys are words and values are their respective counts. + */ + public static Map map(String input) { + Map wordCount = new HashMap<>(); + String[] words = input.split("\\s+"); + for (String word : words) { + word = word.toLowerCase().replaceAll("[^a-z]", ""); + if (!word.isEmpty()) { + wordCount.put(word, wordCount.getOrDefault(word, 0) + 1); + } + } + return wordCount; + } +} diff --git a/map-reduce/src/main/java/com/iluwatar/Reducer.java b/map-reduce/src/main/java/com/iluwatar/Reducer.java new file mode 100644 index 000000000000..6bd9f5f10625 --- /dev/null +++ b/map-reduce/src/main/java/com/iluwatar/Reducer.java @@ -0,0 +1,56 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import java.util.ArrayList; +import java.util.Comparator; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** + * The Reducer class is responsible for aggregating word counts from the shuffled data. + */ +public class Reducer { + private Reducer() { + throw new UnsupportedOperationException("Reducer is a utility class and cannot be instantiated."); + } + /** + * Sums the occurrences of each word and sorts the results in descending order. + * + * @param grouped A map where keys are words and values are lists of their occurrences. + * @return A sorted list of word counts in descending order. + */ + public static List> reduce(Map> grouped) { + Map reduced = new HashMap<>(); + for (Map.Entry> entry : grouped.entrySet()) { + reduced.put(entry.getKey(), entry.getValue().stream().mapToInt(Integer::intValue).sum()); + } + + List> result = new ArrayList<>(reduced.entrySet()); + result.sort(Map.Entry.comparingByValue(Comparator.reverseOrder())); + return result; + } +} diff --git a/map-reduce/src/main/java/com/iluwatar/Shuffler.java b/map-reduce/src/main/java/com/iluwatar/Shuffler.java new file mode 100644 index 000000000000..7eacee034642 --- /dev/null +++ b/map-reduce/src/main/java/com/iluwatar/Shuffler.java @@ -0,0 +1,56 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** + * The Shuffler class is responsible for grouping word occurrences from multiple mappers. + */ +public class Shuffler { + + private Shuffler() { + throw new UnsupportedOperationException("Shuffler is a utility class and cannot be instantiated."); + } + /** + * Merges multiple word count maps into a single grouped map. + * + * @param mapped List of maps containing word counts from the mapping phase. + * @return A map where keys are words and values are lists of their occurrences across inputs. + */ + public static Map> shuffleAndSort(List> mapped) { + Map> grouped = new HashMap<>(); + for (Map map : mapped) { + for (Map.Entry entry : map.entrySet()) { + grouped.putIfAbsent(entry.getKey(), new ArrayList<>()); + grouped.get(entry.getKey()).add(entry.getValue()); + } + } + return grouped; + } +} diff --git a/map-reduce/src/test/java/com/iluwatar/MapReduceTest.java b/map-reduce/src/test/java/com/iluwatar/MapReduceTest.java new file mode 100644 index 000000000000..86dbeff5cb84 --- /dev/null +++ b/map-reduce/src/test/java/com/iluwatar/MapReduceTest.java @@ -0,0 +1,48 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import org.junit.jupiter.api.Test; +import java.util.*; + +import static org.junit.jupiter.api.Assertions.*; + +class MapReduceTest { + + @Test + void testMapReduce() { + List inputs = Arrays.asList( + "Hello world hello", + "MapReduce is fun", + "Hello from the other side" + ); + + List> result = MapReduce.mapReduce(inputs); + + assertEquals("hello", result.get(0).getKey()); // hello = 3 + assertEquals(3, result.get(0).getValue()); + assertEquals(1, result.get(1).getValue()); + } +} diff --git a/map-reduce/src/test/java/com/iluwatar/MapperTest.java b/map-reduce/src/test/java/com/iluwatar/MapperTest.java new file mode 100644 index 000000000000..d8011c9846f3 --- /dev/null +++ b/map-reduce/src/test/java/com/iluwatar/MapperTest.java @@ -0,0 +1,50 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import org.junit.jupiter.api.Test; +import java.util.Map; +import static org.junit.jupiter.api.Assertions.*; + +class MapperTest { + + @Test + void testMapSingleSentence() { + String input = "Hello world hello"; + Map result = Mapper.map(input); + + assertEquals(2, result.get("hello")); + assertEquals(1, result.get("world")); + } + + @Test + void testMapCaseInsensitivity() { + String input = "HeLLo WoRLd hello WORLD"; + Map result = Mapper.map(input); + + assertEquals(2, result.get("hello")); + assertEquals(2, result.get("world")); + } +} diff --git a/map-reduce/src/test/java/com/iluwatar/ReducerTest.java b/map-reduce/src/test/java/com/iluwatar/ReducerTest.java new file mode 100644 index 000000000000..ab4490b9e1a2 --- /dev/null +++ b/map-reduce/src/test/java/com/iluwatar/ReducerTest.java @@ -0,0 +1,74 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import org.junit.jupiter.api.Test; +import java.util.*; + +import static org.junit.jupiter.api.Assertions.*; + +class ReducerTest { + + @Test + void testReduceWithMultipleWords() { + Map> input = new HashMap<>(); + input.put("apple", Arrays.asList(2, 3, 1)); + input.put("banana", Arrays.asList(1, 1)); + input.put("cherry", List.of(4)); + + List> result = Reducer.reduce(input); + + assertEquals(3, result.size()); + assertEquals("apple", result.get(0).getKey()); + assertEquals(6, result.get(0).getValue()); + assertEquals("cherry", result.get(1).getKey()); + assertEquals(4, result.get(1).getValue()); + assertEquals("banana", result.get(2).getKey()); + assertEquals(2, result.get(2).getValue()); + } + + @Test + void testReduceWithEmptyInput() { + Map> input = new HashMap<>(); + + List> result = Reducer.reduce(input); + + assertTrue(result.isEmpty()); + } + + @Test + void testReduceWithTiedCounts() { + Map> input = new HashMap<>(); + input.put("tie1", Arrays.asList(2, 2)); + input.put("tie2", Arrays.asList(1, 3)); + + List> result = Reducer.reduce(input); + + assertEquals(2, result.size()); + assertEquals(4, result.get(0).getValue()); + assertEquals(4, result.get(1).getValue()); + // Note: The order of tie1 and tie2 is not guaranteed in case of a tie + } +} diff --git a/map-reduce/src/test/java/com/iluwatar/ShufflerTest.java b/map-reduce/src/test/java/com/iluwatar/ShufflerTest.java new file mode 100644 index 000000000000..cda651c59a2a --- /dev/null +++ b/map-reduce/src/test/java/com/iluwatar/ShufflerTest.java @@ -0,0 +1,47 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar; + +import org.junit.jupiter.api.Test; +import java.util.*; + +import static org.junit.jupiter.api.Assertions.*; + +class ShufflerTest { + + @Test + void testShuffleAndSort() { + List> mappedData = Arrays.asList( + Map.of("hello", 1, "world", 2), + Map.of("hello", 2, "java", 1) + ); + + Map> grouped = Shuffler.shuffleAndSort(mappedData); + + assertEquals(Arrays.asList(1, 2), grouped.get("hello")); + assertEquals(List.of(2), grouped.get("world")); + assertEquals(List.of(1), grouped.get("java")); + } +} diff --git a/pom.xml b/pom.xml index 5ac889f7f24f..4571118fa81d 100644 --- a/pom.xml +++ b/pom.xml @@ -225,6 +225,7 @@ money table-inheritance bloc + map-reduce