[TODO] flagmining

flagmining is the name of a batteries-not-only-included-but-shoved-up-your-nose personal library I’ve developed for working on CTF tasks I enjoy. In particular it focuses on the kind of problems that would usually be tagged with crypto, misc, math, or ppc.

To give an idea, all my ad-hoc scripts tend to start with from flagmining.all import *, which translates into the following (at the time of writing):

# Common stuff from the Python standard library that I use all the time. Why not
# just become Julia with a billion names in global namespace? At least we don't
# have 1-indexing...
from pathlib import Path
from functools import reduce, singledispatch, lru_cache
from itertools import count, chain, count, starmap, product, zip_longest, combinations
from dataclasses import dataclass
from secrets import token_bytes, token_hex, token_urlsafe
import secrets
from base64 import b64decode, b64encode
from collections import Counter, defaultdict, deque, namedtuple, abc
import logging
from logging import warning, info, debug, error
import math
from math import prod  # Not in gmpy2.
import time
import datetime as dt
import re
import os
from os import urandom
import sys
import ast
import zlib
import random
from random import getrandbits, randrange
import operator as op
import hashlib
import json
import pickle
from timeit import timeit

import numpy as np

# I hate languages rolling their own bigints. Stop, please. GMP is just so far
# ahead of everything else.
import gmpy2

# Some other useful third-party libraries that could be in the kitchensink-like
# Python standard library.
import requests
from PIL import Image
from tqdm import tqdm, trange  # So useful.
# import Crypto

# Sane numpy defaults.
np.set_printoptions(suppress=True, edgeitems=30, threshold=5000, linewidth=400)
# XXX: how to deal with this.

from .monkey_patch import *  # Black arts.
from .utils import *  # Très important.

from .jsdict import *  # The only compliment I'll ever pay JavaScript.
# from .bytes import *  # now in utils.
# from .iterators import *  # now in utils.
# from .compression import *  # MISSING.
# from .automata import *  # TODO: isolate from ~/misc

from .bits import *  # Bits are the atoms of our universe.
from .state import *  # Simple state for standalone scripts.

# The real juice.
from .euler import *
from .primes import *

# Various.
from .subst import *  # substitution boxes and permutations.
from .xor import *
from .digits import *
from .sbox import *
from .oracles import *
from .text import *
from .groups import *
from .code import *  # TODO: move to utils?
from .time import *  # Time utilities.
from .numpy import *  # Numpy extras.

from .lcg import *  # Linear congruential generator.
from .aes import *
from .chacha import *
from .rc4 import *
from .rsa import *
# from .ec import *  # TODO: move from sage scripts after flagrs

from .pbyte import *  # Probabilistic bytes.

# Import the following to automatically run pdb debugger on exception:
# import flagmining.debug

In the beginning, there was chaos. Everything was ad-hoc — ugly scrawlings on the walls of insane asylums — one-off scripts left behind in some random directory, code lost in the REPL history. There was ~/misc, before I hardly knew what CTFs were, when poiko was just feeding me odd problem over chat, later to become ~/misc/ctf. It grew to hundreds of files — tasks, problem data, .tgzs, solve22.py, solveeee.py, solve222221.py, solqwfqwf.py — before I even started making directories. Then there was ~/tmp/ctf and ~/ctf, depending on my involvement with actually playing myself. I had to rg every time I saw a problem I knew I had solved before, praying that I’d named the functions something sane (unlikely). the time I just solve tasks directly in the Python REPL anyway, so I had to also make sure to search that history…

I’m not the most organized. But I am chaotically trying to get better. This is one such effort, attempting to extract common functionality, ergonomics, utilities, and so forth from my problem solving efforts.

I’m far from done, but here I plan to document parts of it, so that it might be useful open-source one day.