How to check file existence using Python?

Small tutorial with code examples and video about how to check if file exists in Python
10 August

There are several Python methods to check does a file exist.

1. os.path.isfile
It returns True if path is an existing regular file. This follows symbolic links, so both islink() and isfile() can be true for the same path.

import os.path

if you need to be sure it's a file.

2. Starting with Python 3.4, the pathlib module offers an object-oriented approach (backported to pathlib2 in Python 2.7):

from pathlib import Path

my_file = Path("/path/to/file")
if my_file.is_file():
    # file exists

3. To check a directory, use:

if my_file.is_dir():
    # directory exists

4. To check whether a Path object exists independently of whether is it a file or directory, use exists():

if my_file.exists():
    # path exists

You can also use resolve() in a try block:

    my_abs_path = my_file.resolve():
except FileNotFoundError:
    # doesn't exist
    # exists

Why tuples use less space in memory than lists in Python?

Small tutorial on basic classes in Python with code examples 
17 October

Let's create a Python tuple:

>>> a = (1,2,3)
>>> a.__sizeof__()

and list

>>> b = [1,2,3]
>>> b.__sizeof__()

Tuple will use less memory than list. Let's figure out why.

 lists are variable-sized while tuples are fixed-size.

So tuples can store the elements directly inside the struct, lists on the other hand need a layer of indirection (it stores a pointer to the elements). This layer of indirection is a pointer, on 64bit systems that's 64bit, hence 8bytes.

But there's another thing that lists do: They over-allocate. Otherwise list.append would be an O(n) operation always - to make it amortized O(1) (much faster) it over-allocates. But now it has to keep track of the allocated size and the filled size (tuples only need to store one size, because allocated and filled size are always identical). That means each list has to store another "size" which on 64bit systems is a 64bit integer, again 8 bytes.

So lists need at least 16 bytes more memory than tuples. Because of the over-allocation. Over-allocation means it allocates more space than needed. However, the amount of over-allocation depends on "how" you create the list and the append/deletion history:

>>> l = [1,2,3]
>>> l.__sizeof__()
>>> l.append(4)  # triggers re-allocation (with over-allocation), because the original list is full
>>> l.__sizeof__()

>>> l = []
>>> l.__sizeof__()
>>> l.append(1)  # re-allocation with over-allocation
>>> l.__sizeof__()
>>> l.append(2)  # no re-alloc
>>> l.append(3)  # no re-alloc
>>> l.__sizeof__()
>>> l.append(4)  # still has room, so no over-allocation needed (yet)
>>> l.__sizeof__()

Some handy links on this topic:

  • tuple struct
  • list struct