pySpoolerRenamer — or how to rename a file stream
Photo by Ankush Minda on Unsplash
Hello my friend!
Working with large data streams and analyzing them, you face a problem determining the ownership of the file, its source, since 99% of all flows generate soulless electric machines.
Sometimes it can be difficult to determine where the sasder345asd.txt or 123_HDR-Tas.csv file came from (in fact, this is just a set of letters, in most cases the file names have a structured name, but this is still hard to read for a person).
Initially, the solution was a small set of bash scripts for elementary renaming of files, everything changed when the number of threads increased dramatically. And a deeper hierarchy of unloading directories appeared, for example:
and using scripts didn't help anymore.
As always, python came to the rescue, in a few hours a script was prepared that coped with this function. After going through several iterations of fixes, improvements and 2 years of work in production, I am ready to provide this utility to the public - pySpoolerRenamer