tcl-mmap

tcl−mmap − POSIX Message Queues for Tcl

package require mmap

mmap ?−shared? ?−length bytes? ?−offset bytes? channelId

This extension provides a Tcl interface to the mmap(2) POSIX system call and associate functions: msync(2) and munmap(2).

In POSIX Operating Systems, the ’mmap’ system call maps a regular file to a memory region. Any operation (read or write) performed on the memory region is reflected back to the mmapped file. The primary advantage of ’mmap’ over regular file access is speed, since memory operations are generally faster and make better use of virtual memory. In fact, the speed−up can be as much as 100% [1].

The functionality of ’mmap’ is exported from this extension in the form of a new Tcl channel type, named "mmap". A memory mapping is established with the ’mmap’ command. Following ’mmap’ execution, access to the memory mapped file is done via the standard Tcl commands: puts/gets/seek/flush/close/fconfigure, only that this time these commands operate on memory, rather than on a file.

The "mmap" channel type is by nature non−blocking, and by default it is in non−buffering mode. It is absolutely essential that they remain always in this mode.

The ’mmap’ command accepts the following parameters:

−shared With this flag, memory changes propagate back to the file. Without this flag changes remain in memory only. This flag when used to map /dev/zero allows for shared memory of the same area between related processes. Default is not shared.

−length The number of bytes of the target file to map into memory. If not set, the whole file is mapped.

−offset File offset to start mapping from.

channelId An open Tcl file channel that points to the file to be mapped.

Returns: A newly created channelId for the requested memory map.

The access pattern of channelId is transfered to the memory map. In other words, if the file is opened "r+", then the memory map will also accept read and write operations; if the file is opened "r" then only read operations are permitted. A write−only file (either "w" or "a") is not acceptable. After ’mmap’ is executed, the associated file can be closed with no effect to the file−memory mapping.

The mapping is released either when the process terminates or the associated file descriptor is closed. Synchronization between memory and file is enforced with every write. However notice that Tcl might buffer data, and hence the file might not remain synchronized with the memory all the time. Use ’flush’ or ’fconfigure’ to set the channel buffering to "none" to guarantee synchronization at all times.

Note that writing to the memory of the mmaped file cannot extent the file itself. Writes at the end of the file automatically loop around the start of it. Reads, on the other hand, that reach end of file, return EOF.

Some possible usages for this extension are:

1. To memory map files for improved access efficiency (read & write).

2. Shared memory between related processes (Tcl interpreters) by memory mapping the same area of /dev/zero.

3. Easily and intuitively implement log files with the following properties:

a. The log file has fixed pre−determined size. Thus a runaway process that continuous logs messages cannot cause system breakdown due to exhaustion of available disk space.

b. The log file always retain the most recent messages that fit within its size.

c. The log file is persistent in the sence that all log operations are reflected directly to the file.

d. The log file is cyclic. When a write would go past the end of the file, the file seek position is initialized to zero, and write continues from the start of the file. (File reads on the other hand don’t loop).

package require mmap

# Open a specific region of the file
set fd [open /var/log/messages r]
set mp [mmap −length 300 −offset 4096 $fd]
close $fd

# Read that region
puts Gets:
for {set i 0} {$i < 10} {incr i} {
    gets $mp line
    puts $line
}
close $mp

# Open for read/write
set fd [open /var/log/messages r+]
set mp [mmap −shared −length 30 $fd]
close $fd

# Use seek to move around the memory mapped file
seek $mp 1
puts −nonewline $mp alex ;# Will overwrite contents

# Force synchronization
flush $mp

# Read the whole file
puts Read:
seek $mp 0
puts [read $mp]

# EOF works as expected
puts [eof $mp]

# Release reserved memory & synchronize
close $mp

Alexandros Stergiakis sterg@kth.se

Copyright (C) 2008 Alexandros Stergiakis

This program is free software: you can redistribute it and/or
modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of
the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.