Dos2unix Command on Linux with Examples

r00t

The Dos2unix command is an essential utility in the Linux environment, designed to convert text files from DOS/MAC format, which uses carriage return and line feed characters for line endings, to Unix format, which uses only a line feed character. This conversion is crucial for ensuring text files are compatible across different operating systems, particularly when transferring files from Windows to Linux systems.

Table of Contents

Understanding DOS and Unix Text Formats

Understanding dos2unix is important for anyone working in mixed Linux/Windows environments, as file transfers between these operating systems can cause issues due to their different text file formats. Mastering dos2unix allows you to effortlessly convert files between formats, ensuring compatibility and avoiding frustrating problems.

Understanding Text File Formats

To understand why dos2unix is needed, we first need to discuss the different text file formats used by DOS/Windows and Unix/Linux.

On DOS/Windows, a line break consists of two characters – a Carriage Return (CR) followed by a Line Feed (LF). In Unix/Linux, a simple Line Feed character is used. So a file moved from Windows to Linux without conversion will have extra CR characters at the end of each line.

Additionally, DOS/Windows uses different encodings like CP-1252 whereas Linux typically uses UTF-8. Without conversion, characters in transferred files can become corrupted.

These differences mean text files formatted for one operating system may not display or process correctly on the other. Conversion with dos2unix is necessary to avoid issues.

Installing Dos2unix

Dos2unix is easy to install from the default repositories on most Linux distributions:

# Debian/Ubuntu 
sudo apt install dos2unix

# RHEL/CentOS
sudo yum install dos2unix 

# Arch Linux
sudo pacman -S dos2unix

Dos2unix Command Basics

The basic syntax of the dos2unix command is:

dos2unix [options] infile [outfile]

This converts the text file “infile” from DOS to Unix format, optionally saving the output to “outfile“.

For example, to convert the file text.txt from Windows to Linux format:

dos2unix text.txt

To convert text.txt but save the Unix/Linux formatted output to a new file called text-linux.txt:

dos2unix text.txt text-linux.txt

This makes it easy to convert files without losing the original DOS/Windows formatted version.

To display the converted file contents instead of saving to a file, pipe to cat:

dos2unix text.txt | cat

To convert all .txt files in a directory:

dos2unix *.txt

Or process text files fed via stdin:

cat text.txt | dos2unix

So in just a few commands, you can convert between text file formats on Linux using dos2unix. Next, we’ll see how to access more advanced functionality with dos2unix options.

Dos2unix Command Options

The dos2unix command includes several options to control conversion behavior:

-ascii   Convert only line breaks (default)
-iso     Conversion between DOS and ISO-8859-1  character set
-7       Convert between DOS and 7-bit ASCII
-u       Convert UTF-16 to UTF-8 Unicode (Linux only)  
-b       Backup original files with .bak extension
-c       Operating system compatible output format
-o       Overwrite original input files 
-q       Quiet mode, suppress all warnings
-n       New file mode, write to output file
-v       Verbose output mode

Let’s look at some usage examples of these key dos2unix options:

Convert a file to 7-bit ASCII only:

dos2unix -7 text.txt

Backup the original file before converting:

dos2unix -b text.txt

Perform Unicode UTF-16 to UTF-8 conversion:

dos2unix -u utf16.txt utf8.txt

Overwrite the original DOS-formatted file after converting:

dos2unix -o text.txt

So by leveraging these options, you can customize dos2unix conversion behavior to suit any scenario.

Comparison With Similar Commands

There are a few other Linux commands that can convert between text file formats:

tr – Low-level translation between single characters
awk – Programming language suited for text processing
iconv – Convert between character encodings
sed – Stream editing for find/replace operations

In particular, tr and awk perform similar functions to dos2unix when it comes to line break conversion.

However, dos2unix has simplified options tailored to line break translation. It also outperforms awk and tr when handling large files.

So while the other commands may be more flexible, dos2unix is optimized for precisely this text file conversion task. It will provide the simplest and most efficient solution in most cases.

Using Dos2unix For Scripting

The dos2unix can be incorporated into Bash scripts and code to automate file format conversions:

Bash

#!/bin/bash

for file in *.txt ; do
  dos2unix "$file"
done

Python

import subprocess

files = ['text1.txt', 'text2.txt']

for f in files:
  subprocess.run(["dos2unix", f])

PHP

<?php

$files = glob("*.txt");

foreach($files as $file) {
  shell_exec("dos2unix $file");  
}

?>

So utilizing dos2unix in code unlocks new possibilities for developers and sysadmins alike.

Conclusion

As we have seen, dos2unix is an invaluable tool for effortlessly converting plaintext files between Windows and Linux formats.

By leveraging all of the knowledge in this guide, from core concepts to advanced troubleshooting, you should have a complete dos2unix skillset. You will be ready to overcome file compatibility issues and seamlessly transfer data between operating systems.

The simple yet dependable dos2unix command should be a standard part of any Linux user’s toolbox. So get out there, start converting your text files, and never worry about Windows/Linux line endings again!

r00t