|
| 1 | +--- |
| 2 | +id: lesson-2 |
| 3 | +title: "Using Memory-Mapped Files" |
| 4 | +sidebar_label: Memory-Mapped Files |
| 5 | +sidebar_position: 2 |
| 6 | +description: "Using Memory-Mapped Files" |
| 7 | +tags: [courses,Advance-level,Introduction] |
| 8 | +--- |
| 9 | + |
| 10 | + |
| 11 | +Memory-mapped files allow a file to be mapped into the memory space of a process. This approach can be efficient for large files and can provide a way to access file contents as if they were in memory. |
| 12 | + |
| 13 | +**Example**: Memory-Mapped Files with `mmap` |
| 14 | + |
| 15 | +```ruby |
| 16 | +require 'mmap' |
| 17 | +
|
| 18 | +File.open('large_file.txt', 'r') do |file| |
| 19 | + length = file.size |
| 20 | + map = Mmap.new(file.fileno, length, Mmap::ACCESS_READ) |
| 21 | +
|
| 22 | + # Access the mapped file |
| 23 | + puts map[0..100] # Read the first 100 bytes |
| 24 | +
|
| 25 | + map.close |
| 26 | +end |
| 27 | +``` |
| 28 | + |
| 29 | +**Note**: The `mmap` gem provides memory-mapped file capabilities. Install it via `gem install mmap`. |
| 30 | + |
| 31 | +#### **File I/O Performance Optimization** |
| 32 | + |
| 33 | +To optimize file I/O performance, consider using buffering and minimizing the number of I/O operations. |
| 34 | + |
| 35 | +**Example**: Buffered Writing |
| 36 | + |
| 37 | +```ruby |
| 38 | +File.open('buffered_output.txt', 'w') do |file| |
| 39 | + file.sync = false # Disable auto-flushing |
| 40 | +
|
| 41 | + (1..1000000).each do |i| |
| 42 | + file.write("Line #{i}\n") |
| 43 | + file.flush if i % 1000 == 0 # Manually flush buffer periodically |
| 44 | + end |
| 45 | +end |
| 46 | +``` |
| 47 | + |
| 48 | +**Example**: Using `IO#readpartial` for Efficient Reading |
| 49 | + |
| 50 | +```ruby |
| 51 | +File.open('large_file.txt', 'r') do |file| |
| 52 | + buffer = "" |
| 53 | + while file.readpartial(1024, buffer) |
| 54 | + # Process the buffer |
| 55 | + puts buffer |
| 56 | + buffer.clear |
| 57 | + end |
| 58 | +end |
| 59 | +``` |
| 60 | + |
| 61 | +:::tip |
| 62 | +1. **Avoid Reading Entire Files into Memory**: |
| 63 | + - For large files, process data in chunks to avoid high memory usage. |
| 64 | + |
| 65 | +2. **Implement Proper File Locking**: |
| 66 | + - Use file locking mechanisms to prevent concurrent access issues, especially in multi-process applications. |
| 67 | + |
| 68 | +3. **Optimize I/O Operations**: |
| 69 | + - Use buffering to reduce the number of I/O operations and improve performance. Consider memory-mapped files for large datasets. |
| 70 | + |
| 71 | +4. **Efficient File Processing**: |
| 72 | + - Process large files efficiently by reading and writing in chunks. Avoid loading entire files into memory. |
| 73 | + |
| 74 | +5. **File Locking**: |
| 75 | + - Use file locking to prevent data corruption in concurrent file access scenarios. |
| 76 | + |
| 77 | +6. **Performance Optimization**: |
| 78 | + - Optimize file I/O operations by using techniques such as buffering and memory-mapped files for better performance. |
| 79 | +::: |
0 commit comments