Using tempfile.mkstemp correctly

The mkstemp function in the tempfile module returns a tuple of 2 values:

• an OS-level handle to an open file (as would be returned by os.open())
• the absolute pathname of that file.

I often see code using mkstemp only to get the filename to the temporary file, following a pattern such as:

from tempfile import mkstemp
import os

def need_temp_storage():
    _, temp_path = mkstemp()
    os.system('some_commande --output %s' % temp_path)
    file = open(temp_path, 'r')
    data = file.read()
    file.close()
    os.remove(temp_path)
    return data

This seems to be working fine, but there is a bug hiding in there. The bug will show up on Linux if you call this functions many time in a long running process, and on the first call on Windows. We have leaked a file descriptor.

The first element of the tuple returned by mkstemp is typically an integer used to refer to a file by the OS. In Python, not closing a file is usually no big deal because the garbage collector will ultimately close the file for you, but here we are not dealing with file objects, but with OS-level handles. The interpreter sees an integer and has no way of knowing that the integer is connected to a file. On Linux, calling the above function repeatedly will eventually exhaust the available file descriptors. The program will stop with:

IOError: [Errno 24] Too many open files: '/tmp/tmpJ6g4Ke'

On Windows, it is not possible to remove a file which is still opened by another process, and you will get:

Windows Error [Error 32]

Fixing the above function requires closing the file descriptor using os.close_():

from tempfile import mkstemp
import os

def need_temp_storage():
    fd, temp_path = mkstemp()
    os.system('some_commande --output %s' % temp_path)
    file = open(temp_path, 'r')
    data = file.read()
    file.close()
    os.close(fd)
    os.remove(temp_path)
    return data

If you need your process to write directly in the temporary file, you don't need to call os.write_(fd, data). The function os.fdopen_(fd) will return a Python file object using the same file descriptor. Closing that file object will close the OS-level file descriptor.