Showing posts with label linux. Show all posts
Showing posts with label linux. Show all posts

Sunday, November 28, 2010

How to split one file into multiple files on linux?

When you have a list of input files that you want to analyze, but the software you are using doesn't support paralelization, a convenient way of speeding up the analysis is by running the program on a subsets of your files at the same time.
If you have a list of input files in one file (input.files.txt), it can easily be split into multiple files using the linux split command.

split -a 2 -l 500 -d input.files.txt input.file.

which will produce files named:

input.file.00
input.file.01
...
input.file.99

with each file containing 500 rows.