dorkhub
show your thing

ugrep

🔍 ugrep 7.8 file pattern searcher -- a user-friendly, faster, more capable grep replacement. Includes a TUI, Google-li…

Genivia
C++★ 3.2k⑂ 140 forksBSD-3-Clauseupdated 1 day ago
code-searchfile-indexingfile-searchfuzzy-searchgrephexdumpinteractiverecursively-search
visit the demogit clone https://github.com/Genivia/ugrep.gitGenivia/ugrep
README.mdfork it — it’s yours

The ugrep file pattern searcher

[ README | User Guide | Indexing | Benchmarks | Q&A ]


option -Q opens a query TUI to search files as you type!

Why use ugrep?

  • ugrep is compatible to GNU/BSD grep, but much faster and with a ton of new features

  • ugrep is a drop-in replacement for BSD and GNU grep (copy or symlink ug to grep, and to egrep and to fgrep), unlike other popular grep claiming to be "grep alternatives" or "replacements" when those actually implement incompatible command-line options and use an incompatible regex matcher, i.e. Perl regex only versus POSIX BRE (grep) and ERE (egrep) when ugrep supports all regex modes

  • ugrep is typically faster than rg, ag, and GNU grep

  • full Unicode extended regex pattern syntax with multi-line pattern matching without any special command-line options

  • includes an interactive TUI with built-in help, Google-like search with AND/OR/NOT patterns, fuzzy search, search (nested) zip/7z/tar/pax/cpio archives, tarballs and compressed files gz/Z/bz/bz2/lzma/xz/lz4/zstd/brotli, search and hexdump binary files, search documents such as PDF, doc, docx, and output in JSON, XML, CSV or your own customized format

  • includes a file indexer to speed up searching slow and cold file systems

Development roadmap

Overview

Commands

  • ug is for interactive use, which loads an optional .ugrep configuration file with your preferences located in the working directory or home directory, ug+ also searches pdfs, documents, e-books, image metadata

  • ugrep for batch use like GNU grep without a .ugrep configuration file, ugrep+ also searches pdfs, documents, e-books, image metadata

What does ugrep add that GNU grep does not support?

  • Matches Unicode patterns by default and automatically searches UTF-8, UTF-16 and UTF-32 encoded files

  • Matches multiple lines with \n or \R in regex patterns, no special options are required to do so!

  • Built-in help: ug --help, where ug --help WHAT displays options related to WHAT you are looking for

    💡 ug --help regex, ug --help globs, ug --help fuzzy, ug --help format.

  • User-friendly with customizable configuration files used by the ug command intended for interactive use that loads a .ugrep configuration file with your preferences

    ug PATTERN ...                         ugrep --config PATTERN ...

    💡 ug --save-config ...options-you-want-to-save... saves a .ugrep config file in the working directory so that the next time you run ug there it uses these options. Do this in your home directory to save a .ugrep config file with options you generally want to use.

  • Interactive query TUI, press F1 or CTRL-Z for help and TAB/SHIFT-TAB to navigate to dirs and files

    ug -Q                                  ug -Q -e PATTERN

    💡 -Q replaces PATTERN on the command line to let you enter patterns interactively in the TUI. In the TUI use ALT+letter keys to toggle short "letter options" on/off, for example ALT-n (option -n) to show/hide line numbers.

  • Search the contents of archives (zip, tar, pax, jar, cpio, 7z) and compressed files (gz, Z, bz, bz2, lzma, xz, lz4, zstd, brotli)

    ug -z PATTERN ...                      ug -z --zmax=2 PATTERN ...

    💡 specify -z --zmax=2 to search compressed files and archives nested within archives. The --zmax argument may range from 1 (default) to 99 for up to 99 decompression and de-archiving steps to search nested archives

  • Search with Google-like Boolean query patterns using -% patterns with AND (or just space), OR (or a bar |), NOT (or a dash -), using quotes to match exactly, and grouping with ( ) (shown on the left side below); or with options -e (as an "or"), --and, --andnot, and --not regex patterns (shown on the right side below):

    ug -% 'A B C' ...                      ug -e 'A' --and 'B' --and 'C' ...
ug -% 'A|B C' ...                      ug -e 'A' -e 'B' --and 'C' ...
ug -% 'A -B -C' ...                    ug -e 'A' --andnot 'B' --andnot 'C' ...
ug -% 'A -(B|C)'...                    ug -e 'A' --andnot 'B' --andnot 'C' ...
ug -% '"abc" "def"' ...                ug -e '\Qabc\E' --and '\Qdef\E' ...

    where A, B and C are arbitrary regex patterns (use option -F to search strings)

    💡 specify option -%% (--bool --files) to apply the Boolean query to files as a whole: a file matches if all Boolean conditions are satisfied by matching patterns file-wide. Otherwise, Boolean conditions apply to single lines by default, since grep utilities are generally line-based pattern matchers. Option --stats displays the query in human-readable form after the search completes.

  • Search pdf, doc, docx, e-book, and more with ug+ using filters associated with filename extensions:

    ug+ PATTERN ...

    or specify --filter with a file type to use a filter utility:

    ug --filter='pdf:pdftotext % -' PATTERN ...
ug --filter='doc:antiword %' PATTERN ...
ug --filter='odt,docx,epub,rtf:pandoc --wrap=preserve -t plain % -o -' PATTERN ...
ug --filter='odt,doc,docx,rtf,xls,xlsx,ppt,pptx:soffice --headless --cat %' PATTERN ...
ug --filter='pem:openssl x509 -text,cer,crt,der:openssl x509 -text -inform der' PATTERN ...
ug --filter='latin1:iconv -f LATIN1 -t UTF-8' PATTERN ...

    💡 the ug+ command is the same as the ug command, but also uses filters to search PDFs, documents, and image metadata

  • Display horizontal context with option -o (--only-matching) and context options -ABC, e.g. to find matches in very long lines, such as Javascript and JSON sources:

    ug -o -C20 -nk PATTERN longlines.js

    💡 -o -C20 fits all matches with context in 20 characters before and 20 charactess after a match (i.e. 40 Unicode characters total), -nk outputs line and column numbers.

  • Find approximate pattern matches with fuzzy search, within the specified Levenshtein distance

    ug -Z PATTERN ...                      ug -Z3 PATTTERN ...

    💡 -Zn matches up to n extra, missing or replaced characters, -Z+n matches up to n extra characters, -Z-n matches with up to n missing characters and -Z~n matches up to n replaced characters. -Z defaults to -Z1.

  • Fzf-like search with regex (or fixed strings with -F), fuzzy matching with up to 4 extra characters with -Z+4 and words only with -w, using -%% for file-wide Boolean searches

    ug -Q -%% -l -w -Z+4 --sort=best

    💡 -l lists the matching files in the TUI, press TAB then ALT-y to view a file, SHIFT-TAB and Alt-l to go back to view the list of matching files ordered by best match

  • Search binary files and display hexdumps with binary pattern matches (Unicode text or -U for byte patterns)

    ug --hexdump -U BYTEPATTERN ...        ug --hexdump TEXTPATTERN ...
ug -X -U BYTEPATTERN ...               ug -X TEXTPATTERN ...
ug -W -U BYTEPATTERN ...               ug -W TEXTPATTERN ...

    💡 --hexdump=4chC1 displays 4 columns of hex without a character column c, no hex spacing h, and with one extra hex line C1 before and after a match.

  • Include files to search by file types or file "magic bytes" or exclude them with ^

    ug -t TYPE PATTERN ...                 ug -t ^TYPE PATTERN ...
ug -M 'MAGIC' PATTERN ...              ug -M '^MAGIC' PATTERN ...

  • Include files and directories to search that match gitignore-style globs or exclude them with ^

    ug -g 'FILEGLOB' PATTERN ...           ug -g '^FILEGLOB' PATTERN ...
ug -g 'DIRGLOB/' PATTERN ...           ug -g '^DIRGLOB/' PATTERN ...
ug -g 'PATH/FILEGLOB' PATTERN ...      ug -g '^PATH/FILEGLOB' PATTERN ...
ug -g 'PATH/DIRGLOB/' PATTERN ...      ug -g '^PATH/DIRGLOB/' PATTERN ...

  • Include files to search by filename extensions (suffix) or exclude them with ^, a shorthand for -g"*.EXT"

    ug -O EXT PATTERN ...                  ug -O ^EXT PATTERN ...

  • Include hidden files (dotfiles) and directories to search (omitted by default)

    ug -. PATTERN ...                      ug -g'.*,.*/' PATTERN ...

    💡 specify hidden in your .ugrep to always search hidden files with ug.

  • Exclude files specified by .gitignore etc.

    ug --ignore-files PATTERN ...          ug --ignore-files=.ignore PATTERN ...

    💡 specify ignore-files in your .ugrep to always ignore them with ug. Add additional ignore-files=... as desired.

  • Search patterns excluding negative patterns ("match this but not that")

    ug -e PATTERN -N NOTPATTERN ...        ug -e '[0-9]+' -N 123 ...

  • Use predefined regex patterns to search source code, javascript, XML, JSON, HTML, PHP, markdown, etc.

    ug PATTERN -f c++/zap_comments -f c++/zap_strings ...
ug PATTERN -f php/zap_html ...
ug -f js/functions ... | ug PATTERN ...

  • Sort matching files by name, best match, size, and time

    ug --sort PATTERN ...                  ug --sort=size PATTERN ...
ug --sort=changed PATTERN ...          ug --sort=created PATTERN ...
ug -Z --sort=best PATTERN ...          ug --no-sort PATTERN ...

  • Output results in CSV, JSON, XML, and user-specified formats

    ug --csv PATTERN ...                   ug --json PATTERN ...
ug --xml PATTERN ...                   ug --format='file=%f line=%n match=%O%~' PATTERN ...

    💡 ug --help format displays help on format % fields for customized output.

  • Search with PCRE's Perl-compatible regex patterns and display or replace subpattern matches

    ug -P PATTERN ...                      ug -P --format='%1 and %2%~' 'PATTERN(SUB1)(SUB2)' ...

  • Replace patterns in the output with -P and --replace replacement text, optionally containing % formatting fields, using -y to pass the rest of the file through:

    ug --replace='TEXT' PATTERN ...        ug -y --replace='TEXT' PATTERN ...
ug --replace='(%m:%o)' PATTERN ...     ug -y --replace='(%m:%o)' PATTERN ...
ug -P --replace='%1' PATTERN ...       ug -y -P --replace='%1' PATTERN ...

    💡 ug --help format displays help on format % fields to optionally use with --replace.

  • Search files with a specific encoding format such as ISO-8859-1 thru 16, CP 437, CP 850, MACROMAN, KOI8, etc.

    ug --encoding=LATIN1 PATTERN ...

Table of contents

How to install

MacOS

Install the latest ugrep with Homebrew:

$ brew install ugrep

or install with MacPorts:

$ sudo port install ugrep

This installs the ugrep and ug commands, where ug is the same as ugrep but also loads the configuration file .ugrep when present in the working directory or home directory.

Windows

Install with Winget winget install Genivia.ugrep

Or install with Chocolatey choco install ugrep

Or install with Scoop scoop install ugrep

Or download the full-featured ugrep.exe executable as release artifact from https://github.com/Genivia/ugrep/releases. The zipped release contains the main ugrep.exe binary as well as ug.exe. The ug command, intended for interactive use, loads and reads in settings from the .ugrep configuration file (when present in the working directory or home directory).

Add ugrep.exe and ug.exe to your execution path: go to Settings and search for "Path" in Find a Setting. Select environment variables -> Path -> New and add the directory where you placed the ugrep.exe and ug.exe executables.

Tip

Practical hints on using ugrep.exe and ug.exe on the Windows command line:

  • when quoting patterns and arguments on the command line, do not use single ' quotes but use " instead; most Windows command utilities consider the single ' quotes part of the command-line argument!
  • file and directory globs are best specified with option -g/GLOB instead of the usual GLOB command line arguments to select files and directories to search, especially for recursive searches;
  • when specifying an empty pattern "" to match all input, this may be ignored by some Windows command interpreters such as Powershell, in that case you must specify option --match instead;
  • to match newlines in patterns, you may want to use \R instead of \n to match any Unicode newlines, such as \r\n pairs and single \r and \n.

Alpine Linux

$ apk add ugrep ugrep-doc

Check https://pkgs.alpinelinux.org/packages?name=ugrep for version info.

Arch Linux

$ pacman -S ugrep

Check https://archlinux.org/packages/extra/x86_64/ugrep for version info.

CentOS

First enable the EPEL repository, then you can install ugrep.

$ dnf install ugrep

Check https://packages.fedoraproject.org/pkgs/ugrep/ugrep/ for version info.

Debian

$ apt-get install ugrep

Check https://packages.debian.org/ugrep for version info. To build and try ugrep locally, see "All platforms" build steps further below.

Fedora

$ dnf install ugrep

Check https://packages.fedoraproject.org/pkgs/ugrep/ugrep/ for version info.

Gentoo

$ emerge sys-apps/ugrep

Check https://packages.gentoo.org/packages/sys-apps/ugrep for version info.

FreeBSD

$ pkg install ugrep

Check https://www.freshports.org/textproc/ugrep for version info.

Haiku

$ pkgman install cmd:ugrep

Check https://github.com/haikuports/haikuports/tree/master/app-text/ugrep for version info. To build and try ugrep locally, see "All platforms" build steps further below.

NetBSD

You can use the standard NetBSD package installer (pkgsrc): http://cdn.netbsd.org/pub/pkgsrc/current/pkgsrc/textproc/ugrep/README.html

OpenBSD

$ pkg_add ugrep

Check https://openports.pl/path/sysutils/ugrep for version info.

OpenSUSE

$ zypper install ugrep

Check https://build.opensuse.org/package/show/utilities/ugrep for version info.

RHEL

First enable the EPEL repository, then you can install ugrep.

$ dnf install ugrep

Check https://packages.fedoraproject.org/pkgs/ugrep/ugrep/ for version info.

Other platforms: step 1 download

Clone ugrep with

$ git clone https://github.com/Genivia/ugrep

Or visit https://github.com/Genivia/ugrep/releases to download a specific release.

Other platforms: step 2 consider optional dependencies

You can always add these later, when you need these features:

  • Option -P (Perl regular expressions) requires either the PCRE2 library (recommended) or the Boost.Regex library (optional fallback). If PCRE2 is not installed, install PCRE2 with e.g. sudo apt-get install -y libpcre2-dev or download PCRE2 and follow the installation instructions. Alternatively, download Boost.Regex and run ./bootstrap.sh and sudo ./b2 --with-regex install. See Boost: getting started.

  • Option -z (compressed files and archives search) requires the zlib library installed. It is installed on most systems. If not, install it, e.g. with sudo apt-get install -y libz-dev. To search .bz and .bz2 files, install the bzip2 library (recommended), e.g. with sudo apt-get install -y libbz2-dev. To search .lzma and .xz files, install the lzma library (recommended), e.g. with sudo apt-get install -y liblzma-dev. To search .lz4 files, install the lz4 library (optional, not required), e.g. with sudo apt-get install -y liblz4-dev. To search .zst files, install the zstd library (optional, not required), e.g. with sudo apt-get install -y libzstd-dev. To search .br files, install the brotli library (optional, not required), e.g. with sudo apt-get install -y libbrotli-dev. To search .bz3 files, install the bzip3 library (optional, not required), e.g. with sudo apt-get install -y bzip3.

Tip

Even if your system has command line utilities, such as bzip2, that does not necessarily mean that the development libraries such as libbz2 are installed. The development libraries should be installed.

Some Linux systems may not be configured to load dynamic libraries from /usr/local/lib, causing a library load error when running ugrep. To correct this, add export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" to your ~/.bashrc file. Or run sudo ldconfig /usr/local/lib.

Other platforms: step 3 build

Execute the ./build.sh script to build ugrep:

$ cd ugrep
$ ./build.sh

This builds the ugrep executable in the ugrep/src directory with ./configure and make -j, verified with make test. When all tests pass, the ugrep executable is copied to ugrep/bin/ugrep and the symlink ugrep/bin/ug -> ugrep/bin/ugrep is added for the ug command.

Note that ug is the same as ugrep but also loads the configuration file .ugrep when present in the working directory or home directory. This means that you can define your default options for ug in .ugrep.

Alternative paths to installed or local libraries may be specified with ./build.sh. To get help on the available build options:

$ ./build.sh --help

You can build static executables by specifying:

$ ./build.sh --enable-static

This may fail if libraries don't link statically, such as brotli. In that case try ./build.sh --enable-static --without-brotli.

You can build ugrep with customized defaults enabled, such as a pager:

$ ./build.sh --enable-pager

Options to select defaults for builds include:

  • --help display build options
  • --enable-static build static executables, if possible
  • --enable-hidden always search hidden files and directories
  • --enable-pager always use a pager to display output on terminals
  • --enable-pretty colorize output to terminals and add filename headings
  • --disable-auto-color disable automatic colors, requires ugrep option --color=auto to show colors
  • --disable-mmap disable memory mapped files
  • --disable-sse2 disable SSE2 and AVX optimizations
  • --disable-avx2 disable AVX2 and AVX512BW optimizations, but compile with SSE2 when supported
  • --disable-neon disable ARM NEON/AArch64 optimizations
  • --with-grep-path the default -f path if GREP_PATH is not defined
  • --with-grep-colors the default colors if GREP_COLORS is not defined

After the build completes, copy ugrep/bin/ugrep and ugrep/bin/ug to a convenient location, for example in your ~/bin directory. Or, if you may want to install the ugrep and ug commands and man pages:

$ sudo make install

This also installs the pattern files with predefined patterns for option -f at /usr/local/share/ugrep/patterns/. Option -f first checks the working directory for the presence of pattern files, if not found checks environment variable GREP_PATH to load the pattern files, and if not found reads the installed predefined pattern files.

Troubleshooting

Git and timestamps

Unfortunately, git clones do not preserve timestamps which means that you may run into "WARNING: 'aclocal-1.15' is missing on your system." or that autoheader was not found when running make.

To work around this problem, run:

$ autoreconf -fi
$ ./build.sh

Compiler warnings

GCC 8 and greater may produce warnings of the sort "note: parameter passing for argument ... changed in GCC 7.1". These warnings should be ignored.

Dockerfile for developers

A Dockerfile is included to build ugrep in a Ubuntu container.

Developers may want to use sanitizers to verify the ugrep code when making significant changes, for example to detect data races with the ThreadSanitizer:

$ ./build.sh CXXFLAGS='-fsanitize=thread -O1 -g'

We checked ugrep with the clang AddressSanitizer, MemorySanitizer, ThreadSanitizer, and UndefinedBehaviorSanitizer. These options incur significant runtime overhead and should not be used for the final build.

🔝 Back to table of contents

Performance comparisons

Please note that the ugrep and ug commands search binary files by default and do not ignore .gitignore specified files, which will not make recursive search performance comparisons meaningful unless options -I and --ignore-files are used. To make these options the default for ug, simply add ignore-binary and ignore-files to your .ugrep configuration file.

For an up-to-date performance comparison of the latest ugrep, please see the ugrep performance benchmarks. Ugrep is faster than GNU grep, Silver Searcher, ack, sift. Ugrep's speed beats ripgrep in most benchmarks.

Using ugrep within Vim

First, let's define the :grep command in Vim to search files recursively. To do so, add the following lines to your .vimrc located in the root directory:

if executable('ugrep')
    set grepprg=ugrep\ -RInk\ -j\ -u\ --tabs=1\ --ignore-files
    set grepformat=%f:%l:%c:%m,%f+%l+%c+%m,%-G%f\\\|%l\\\|%c\\\|%m
endif

This specifies -j case insensitive searches with the Vim :grep command. For case sensitive searches, remove \ -j from grepprg. Multiple matches on the same line are listed in the quickfix window separately. If this is not desired, remove \ -u from grepprg. With this change, only the first match on a line is shown. Option --ignore-files skips files specified in .gitignore files, when present. To limit the depth of recursive searches to the current directory only, append \ -1 to grepprg.

You can now invoke the Vim :grep command in Vim to search files on a specified PATH for PATTERN matches:

:grep PATTERN [PATH]

If you omit PATH, then the working directory is searched. Use % as PATH to search only the currently opened file in Vim:

:grep PATTERN %

The :grep command shows the results in a quickfix window that allows you to quickly jump to the matches found.

To open a quickfix window with the latest list of matches:

:copen

Double-click on a line in this window (or select a line and press ENTER) to jump to the file and location in the file of the match. Enter commands :cn and :cp to jump to the next or previous match, respectively. To update the search results in the quickfix window, just grep them. For example, to recursively search C++ source code marked FIXME in the working directory:

:grep -tc++ FIXME

To close the quickfix window:

:cclose

You can use ugrep options with the :grep command, for example to select single- and multi-line comments in the current file:

:grep -f c++/comments %

Only the first line of a multi-line comment is shown in quickfix, to save space. To show all lines of a multi-line match, remove %-G from grepformat.

A popular Vim tool is ctrlp.vim, which is installed with:

$ cd ~/.vim
$ git clone https://github.com/kien/ctrlp.vim.git bundle/ctrlp.vim

CtrlP uses ugrep by adding the following lines to your .vimrc:

if executable('ugrep')
    set runtimepath^=~/.vim/bundle/ctrlp.vim
    let g:ctrlp_match_window='bottom,order:ttb'
    let g:ctrlp_user_command='ugrep "" %s -Rl -I --ignore-files -3'
endif

where -I skips binary files, option --ignore-files skips files specified in .gitignore files, when present, and option -3 restricts searching directories to three levels (the working directory and up to two levels below).

Start Vim then enter the command:

:helptags ~/.vim/bundle/ctrlp.vim/doc

To view the CtrlP documentation in Vim, enter the command:

:help ctrlp.txt

🔝 Back to table of contents

Using ugrep within Emacs

Thanks to Manuel Uberti, you can now use ugrep in Emacs. To use ugrep instead of GNU grep within Emacs, add the following line to your .emacs.d/init.el file:

(setq-default xref-search-program ‘ugrep)

This means that Emacs commands such as project-find-regexp that rely on Xref can now leverage the power of ugrep.

Furthermore, it is possible to use grep in the Emacs grep commands. For instance, you can run lgrep with ugrep by customizing grep-template to something like the following:

(setq-default grep-template "ugrep --color=always -0Iinr -e <R>")

If you do not have Emacs version 29 (or greater) you can download and build Emacs from the Emacs master branch, or enable Xref integration with ugrep manually:

(with-eval-after-load 'xref
 (push '(ugrep . "xargs -0 ugrep <C> --null -ns -e <R>")
       xref-search-program-alist)
 (setq-default xref-search-program 'ugrep))

🔝 Back to table of contents

Using ugrep to replace GNU/BSD grep

ugrep supports all standard GNU/BSD grep command-line options and improves many of them too. See notable improvements over grep.

In fact, executing ugrep with options -U, -Y, -. and --sort makes it behave like egrep, permitting empty patterns to match and search hidden files instead of ignoring them. See grep equivalence.

  • You can create convenient grep aliases with or without options -Y, -. and --sort or include other options as desired. If you really must stick exactly to GNU/BSD grep ASCII/LATIN1 patterns, use options -U and --grep to disable Unicode pattern matching and to reassign options -z and -Z to --null-data and --null, respectively.

  • You can also create grep, egrep and fgrep executables by symlinking or copying ugrep to those names. When the ugrep (or ugrep.exe) executable is copied as grep (grep.exe), egrep (egrep.exe), fgrep (fgrep.exe), then options -Y and -. are automatically enabled together with either -G for grep, -E for egrep and -F for fgrep. In addition, when copied as zgrep, zegrep and zfgrep, option --decompress is enabled. For example, when ugrep is copied as zegrep, options --decompress, -E, -Y, -. and --sort are enabled.

  • Likewise, symlinks and hard links can be used to create grep, egrep and fgrep replacements in the usual installation directories. For example:

    sudo ln -s `which ugrep` /opt/local/bin/grep
sudo ln -s `which ugrep` /opt/local/bin/egrep
sudo ln -s `which ugrep` /opt/local/bin/fgrep
sudo ln -s `which ugrep` /opt/local/bin/zgrep
sudo ln -s `which ugrep` /opt/local/bin/zegrep
sudo ln -s `which ugrep` /opt/local/bin/zfgrep

    The /opt/local/bin here is an example and may or may not be in your $path and may or may not be found, so please adjust as necessary. Caution: bash does not obey the linked name when executing the program, reverting to the name ugrep instead, which negates all internal compatibility settings. To avoid this, copy the executables instead of linking!

When linking or copying ugrep to grep, egrep, fgrep, zgrep, zegrep, zfgrep, options -z and -Z are reassigned for compatibility to GNU/BSD grep options --null-data and --null, respectively.

Equivalence to GNU/BSD grep

When the ugrep executable file is symlinked or copied to grep, egrep, fgrep, zgrep, zegrep and zfgrep executables, then those executables will behave as GNU grep equivalents. This behavior is implicit and automatic, essentially using the following translations:

grep   = ugrep -G -Y -. --sort
egrep  = ugrep -E -Y -. --sort
fgrep  = ugrep -F -Y -. --sort

zgrep  = ugrep -z -G -Y -. --sort
zegrep = ugrep -z -E -Y -. --sort
zfgrep = ugrep -z -F -Y -. --sort

please note that:

  • -Y enables empty matches, so for example the pattern a* matches every line instead of a sequence of a's. By default in ugrep, the pattern a* matches a sequence of a's. Moreover, in ugrep the pattern a*b*c* matches what it is supposed to match by default. See improvements.
  • -. searches hidden files (dotfiles). By default, hidden files are ignored, like most Unix utilities.
  • --sort specifies output sorted by pathname, showing sorted matching files first followed by sorted recursive matches in subdirectories. Otherwise, matching files are reported in no particular order to improve performance;
  • options -z and -Z are reassigned to --null-data and --null and no longer enable --decompress and --fuzzy searching modes.

There is one minor difference with GNU/BSD grep:

  • GNU/BSD grep defaults to -Dread and -dread which are not recommended, see improvements for an explanation.

🔝 Back to table of contents

Short and quick command aliases

Commonly-used aliases to add to .bashrc to increase productivity:

alias uq     = 'ug -Q'                # interactive TUI search (uses .ugrep config)
alias uz     = 'ug -z'                # compressed files and archives search (uses .ugrep config)
alias ux     = 'ug -U --hexdump'      # binary pattern search (uses .ugrep config)

alias ugit   = 'ug -R --ignore-files' # works like git-grep & define your preferences in .ugrep config

alias grep   = 'ug -G'                # search with basic regular expressions (BRE) like grep
alias egrep  = 'ug -E'                # search with extended regular expressions (ERE) like egrep
alias fgrep  = 'ug -F'                # find string(s) like fgrep
alias zgrep  = 'ug -zG'               # search compressed files and archives with BRE
alias zegrep = 'ug -zE'               # search compressed files and archives with ERE
alias zfgrep = 'ug -zF'               # find string(s) in compressed files and/or archives

alias xdump  = 'ugrep -X ""'                 # hexdump files without searching (don't use .ugrep config)
alias zmore  = 'ugrep+ -z -I -+ --pager ""'  # view compressed, archived and regular files (don't use .ugrep config)

🔝 Back to table of contents

Notable improvements over grep
  • ugrep starts an interactive query TUI with option -Q.
  • ugrep matches patterns across multiple lines when patterns match \n.
  • ugrep matches full Unicode by default (disabled with option -U).
  • ugrep supports Boolean patterns with AND, OR and NOT (option --bool).
  • ugrep supports gitignore with option --ignore-files.
  • ugrep supports fuzzy (approximate) matching with option -Z.
  • ugrep supports user-defined global and local configuration files.
  • ugrep searches compressed files and archives with option -z.
  • ugrep searches cpio, jar, pax, tar, zip and 7z archives with option -z.
  • ugrep searches cpio, jar, pax, tar, zip and 7z archives recursively stored within archives with -z and --zmax=NUM for up to NUM levels deep.
  • ugrep searches pdf, doc, docx, xls, xlsx, epub, and more with --filter using third-party format conversion utilities as plugins.
  • ugrep searches a directory when the FILE argument is a directory, like most Unix/Linux utilities; use option -r to search directories recursively.
  • ugrep does not match hidden files by default like most Unix/Linux utilities (hidden dotfile file matching is enabled with -.).
  • ugrep regular expression patterns are more expressive than GNU grep and BSD grep POSIX ERE and support Unicode pattern matching. Extended regular expression (ERE) syntax is the default (i.e. option -E as egrep, whereas -G enables BRE).
  • ugrep spawns threads to search files concurrently to improve search speed (disabled with option -J1).
  • ugrep produces hexdumps with -W (output binary matches in hex with text matches output as usual) and -X (output all matches in hex).
  • ugrep can output matches in JSON, XML, CSV and user-defined formats (with option --format).
  • ugrep option -f uses GREP_PATH environment variable or the predefined patterns installed in /usr/local/share/ugrep/patterns. If -f is specified and also one or more -e patterns are specified, then options -F, -x, and -w do not apply to -f patterns. This is to avoid confusion when -f is used with predefined patterns that may no longer work properly with these options.
  • ugrep options -O, -M, and -t specify file extensions, file signature magic byte patterns, and predefined file types, respectively. This allows searching for certain types of files in directory trees, for example with recursive search options -R and -r. Options -O, -M, and -t also applies to archived files in cpio, jar, pax, tar, zip and 7z files.
  • ugrep option -k, --column-number to display the column number, taking tab spacing into account by expanding tabs, as specified by option --tabs.
  • ugrep option -P (Perl regular expressions) supports backreferences (with --format) and lookbehinds, which uses the PCRE2 or Boost.Regex library for fast Perl regex matching with a PCRE-like syntax.
  • ugrep option -b with option -o or with option -u, ugrep displays the exact byte offset of the pattern match instead of the byte offset of the start of the matched line reported by GNU/BSD grep.
  • ugrep option -u, --ungroup to not group multiple matches per line. This option displays a matched input line again for each additional pattern match on the line. This option is particularly useful with option -c to report the total number of pattern matches per file instead of the number of lines matched per file.
  • ugrep option -Y enables matching empty patterns. Grepping with empty-matching patterns is weird and gives different results with GNU grep versus BSD grep. Empty matches are not output by ugrep by default, which avoids making mistakes that may produce "random" results. For example, with GNU/BSD grep, pattern a* matches every line in the input, and actually matches xyz three times (the empty transitions before and between the x, y, and z). Allowing empty matches requires ugrep option -Y. Patterns that start with ^ or end with $, such as ^\h*$, match empty. These patterns automatically enable option -Y.
  • ugrep option -D, --devices=ACTION is skip by default, instead of read. This prevents unexpectedly hanging on named pipes in directories that are recursively searched, as may happen with GNU/BSD grep that read devices by default.
  • ugrep option -d, --directories=ACTION is skip by default, instead of read. By default, directories specified on the command line are searched, but not recursively deeper into subdirectories.
  • ugrep offers negative patterns -N PATTERN, which are patterns of the form (?^X) that skip all X input, thus removing X from the search. For example, negative patterns can be used to skip strings and comments when searching for identifiers in source code and find matches that aren't in strings and comments. Predefined zap patterns use negative patterns, for example, use -f cpp/zap_comments to ignore pattern matches in C++ comments.
  • ugrep ignores the GREP_OPTIONS environment variable, because the behavior of ugrep must be portable and predictable on every system. Also GNU grep abandoned GREP_OPTIONS for this reason. Please use the ug command that loads the .ugrep configuration file located in the working directory or in the home directory when present, or use shell aliases to create new commands with specific search options.

🔝 Back to table of contents

Tutorial

Examples

To perform a search using a configuration file .ugrep placed in the working directory or home directory (note that ug is the same as ugrep --config):

ug PATTERN FILE...

To save a .ugrep configuration file to the working directory, then edit this file in your home directory to customize your preferences for ug defaults:

ug --save-config

To search the working directory and recursively deeper for main (note that -r recurse symlinks is enabled by default if no file arguments are specified):

ug main

Same, but only search C++ source code files recursively, ignoring all other files:

ug -tc++ main

Same, using the interactive query TUI, starting with the initial search pattern main (note that -Q with an initial pattern requires option -e because patterns are normally specified interactively and all command line arguments are considered files/directories):

ug -Q -tc++ -e main

To search for #define (and # define etc) using a regex pattern in C++ files (note that patterns should be quoted to prevent shell globbing of * and ?):

ug -tc++ '#[\t ]*define'

To search for main as a word (-w) recursively without following symlinks (-r) in directory myproject, showing the matching line (-n) and column (-k) numbers next to the lines matched:

ug -r -nkw main myproject

Same, but only search myproject without recursing deeper (note that directory arguments are searched at one level by default):

ug -nkw main myproject

Same, but search myproject and one subdirectory level deeper (two levels) with -2:

ug -2 -nkw main myproject

Same, but only search C++ files in myproject and its subdirectories with -tc++:

ug -tc++ -2 -nkw main myproject

Same, but also search inside archives (e.g. zip and tar files) and compressed files with -z:

ug -z -tc++ -2 -nkw main myproject

Search recursively the working directory for main while ignoring gitignored files (e.g. assuming .gitignore is in the working directory or below):

ug --ignore-files -tc++ -nkw main

To list all files in the working directory and deeper that are not ignored by .gitignore file(s):

ug --ignore-files -l ''

To display the list of file name extensions and "magic bytes" (shebangs) that are searched corresponding to -t arguments:

ug -tlist

To list all shell files recursively, based on extensions and shebangs with -l (note that '' matches any non-empty file):

ug -l -tShell ''

🔝 Back to table of contents

Advanced examples

To search for main in source code while ignoring strings and comment blocks you can use negative patterns with option -N to skip unwanted matches in C/C++ quoted strings and comment blocks:

ug -r -nkw -e 'main' -N '"(\\.|\\\r?\n|[^\\\n"])*"|//.*|/\*(.*\n)*?.*\*+\/' myproject

This is a lot of work to type in correctly! If you are like me, I don't want to spend time fiddling with regex patterns when I am working on something more important. There is an easier way by using ugrep's predefined patterns (-f) that are installed with the ugrep tool:

ug -r -nkw 'main' -f c/zap_strings -f c/zap_comments myproject

This query also searches through other files than C/C++ source code, like READMEs, Makefiles, and so on. We're also skipping symlinks with -r. So let's refine this query by selecting C/C++ files only using option -tc,c++ and include symlinks to files and directories with -R:

ug -R -tc,c++ -nkw 'main' -f c/zap_strings -f c/zap_comments myproject

What if you only want to look for the identifier main but not as a function main(? In this case, use a negative pattern for this to skip unwanted main\h*( pattern matches:

ug -R -tc,c++ -nkw -e 'main' -N 'main\h*\(' -f c/zap_strings -f c/zap_comments myproject

This uses the -e and -N options to explicitly specify a pattern and a negative pattern, respectively, which is essentially forming the pattern main|(?^main\h*\(), where \h matches space and tab. In general, negative patterns are useful to filter out pattern matches that we are not interested in.

As another example, let's say we may want to search for the word FIXME in C/C++ comment blocks. To do so we can first select the comment blocks with ugrep's predefined c/comments pattern AND THEN select lines with FIXME using a pipe:

ug -R -tc,c++ -nk -f c/comments myproject | ug -w 'FIXME'

Filtering results with pipes is generally easier than using AND-OR logic that some search tools use. This approach follows the Unix spirit to keep utilities simple and use them in combination for more complex tasks.

Let's produce a sorted list of all identifiers found in Java source code while skipping strings and comments:

ug -R -tjava -f java/names myproject | sort -u

This matches Java Unicode identifiers using the regex \p{JavaIdentifierStart}\p{JavaIdentifierPart}* defined in patterns/java/names.

With traditional grep and grep-like tools it takes great effort to recursively search for the C/C++ source file that defines function qsort, requiring something like this:

ug -R --include='*.c' --include='*.cpp' '^([ \t]*[[:word:]:*&]+)+[ \t]+qsort[ \t]*\([^;\n]+$' myproject

Fortunately, with ugrep we can simply select all function definitions in files with extension .c or .cpp by using option -Oc,cpp and by using a predefined pattern functions that is installed with the tool to produce all function definitions. Then we select the one we want:

ug -R -Oc,cpp -nk -f c/functions | ug 'qsort'

Note that we could have used -tc,c++ to select C/C++ files, but this also includes header files when we want to only search .c and .cpp files.

We can also skip files and directories from being searched that are defined in .gitignore. To do so we use --ignore-files to exclude any files and directories from recursive searches that match the globs in .gitignore, when one or more .gitignore files are found:

ug -R -tc++ --ignore-files -f c++/defines

This searches C++ files (-tc++) in the working directory for #define lines (-f c++/defines), while skipping files and directories declared in .gitignore. If you find this too long to type then define an alias to search GitHub directories:

alias ugit='ugrep -R --ignore-files'
ugit -tc++ -f c++/defines

To highlight matches when pushed through a chain of pipes we should use --color=always:

ugit --color=always -tc++ -f c++/defines | ugrep -w 'FOO.*'

This returns a color-highlighted list of all #define FOO... macros in C/C++ source code files, skipping files defined in .gitignore.

Note that the complement of --exclude is not --include, because exclusions always take precedence over inclusions, so we cannot reliably list the files that are ignored with --include-from='.gitignore'. Only files explicitly specified with --include and directories explicitly specified with --include-dir are visited. The --include-from from lists globs that are considered both files and directories to add to --include and --include-dir, respectively. This means that when directory names and directory paths are not explicitly listed in this file then it will not be visited using --include-from.

Because ugrep checks if the input is valid UTF-encoded Unicode (unless -U is used), it is possible to use it as a filter to ignore non-UTF output produced by a program:

program | ugrep -I ''

If the program produces valid output then the output is passed through, otherwise the output is filtered out option -I. If the output is initially valid for a very large portion but is followed by invalid output, then ugrep may initially show the output up to but excluding the invalid output after which further output is blocked.

To filter lines that are valid ASCII or UTF-encoded, while removing lines that are not:

program | ugrep '[\p{Unicode}--[\n]]+'

Note that \p{Unicode} matches \n but we don't want to matche the whole file! Just lines with [\p{Unicode}--[\n]]+.

🔝 Back to table of contents

Displaying helpful info

The ugrep man page:

man ugrep

To show a help page:

ug --help

To show options that mention WHAT:

ug --help WHAT

To show a list of -t TYPES option values:

ug -tlist

In the interactive query TUI, press F1 or CTRL-Z for help and options:

ug -Q

🔝 Back to table of contents

Configuration files

 
--config[=FILE], ---[FILE]
        Use configuration FILE.  The default FILE is `.ugrep'.  The working
        directory is checked first for FILE, then the home directory.  The
        options specified in the configuration FILE are parsed first,
        followed by the remaining options specified on the command line.
        The ug command automatically loads a `.ugrep' configuration file,
        unless --config=FILE or --no-config is specified.
--no-config
        Do not load the default .ugrep configuration file.
--save-config[=FILE] [OPTIONS]
        Save configuration FILE to include OPTIONS.  Update FILE when
        first loaded with --config=FILE.  The default FILE is `.ugrep',
        which is automatically loaded by the ug command.  When FILE is a
        `-', writes the configuration to standard output.  Only part of the
        OPTIONS are saved that do not cause searches to fail when combined
        with other options.  Additional options may be specified by editing
        the saved configuration file.  A configuration file may be modified
        manually to specify one or more config[=FILE] to indirectly load
        the specified FILEs, but recursive config loading is not allowed.

The ug command versus the ugrep command

The ug command is intended for context-dependent interactive searching and is equivalent to the ugrep --config command to load the configuration file .ugrep when present in the working directory or, when not found, in the home directory:

ug PATTERN ...
ugrep --config PATTERN ...

The ug command also sorts files by name per directory searched. A configuration file contains NAME=VALUE pairs per line, where NAME is the name of a long option (without --) and =VALUE is an argument, which is optional and may be omitted depending on the option. Empty lines and lines starting with a # are ignored:

# Color scheme
colors=cx=hb:ms=hiy:mc=hic:fn=hi+y+K:ln=hg:cn=hg:bn=hg:se=
# Disable searching hidden files and directories
no-hidden
# ignore files specified in .ignore and .gitignore in recursive searches
ignore-files=.ignore
ignore-files=.gitignore

Command line options are parsed in the following order: first the (default or named) configuration file is loaded, then the remaining options and arguments on the command line are parsed.

Option --stats displays the configuration file used after searching.

Named configuration files

Named configuration files are intended to streamline custom search tasks, by reducing the number of command line options to just one ---FILE to use the collection of options specified in FILE. The --config=FILE option and its abbreviated form ---FILE load the specified configuration file located in the working directory or, when not found, located in the home directory:

ug ---FILE PATTERN ...
ugrep ---FILE PATTERN ...

An error is produced when FILE is not found or cannot be read.

Named configuration files can be used to define a collection of options that are specific to the requirements of a task in the development workflow of a project. For example to report unresolved issues by checking the source code and documentation for comments with FIXME and TODO items. Such named configuration file can be localized to a project by placing it in the project directory, or it can be made global by placing it in the home directory. For visual feedback, a color scheme specific to this task can be specified with option colors in the configuration FILE to help identify the output produced by a named configuration as opposed to the default configuration.

Saving a configuration file

The --save-config option saves a .ugrep configuration file to the working directory using the current configuration loaded with --config. This saves the current configuration combined with additional options when specified also. Only those options that cannot conflict with other options and options that cannot negatively impact search results will be saved.

The --save-config=FILE option saves the configuration to the specified FILE. The configuration is written to standard output when FILE is a -.

Alternatively, a configuration file may be manually created or modified. A configuration file may include one or more config[=FILE] to indirectly load the specfified FILE, but recursive config loading is prohibited. The simplest way to manuall create a configuration file is to specify config at the top of the file, followed by the long options to override the defaults.

🔝 Back to table of contents

Interactive search with -Q

 
-Q[=DELAY], --query[=DELAY]
        Query mode: start a TUI to perform interactive searches.  This mode
        requires an ANSI capable terminal.  An optional DELAY argument may
        be specified to reduce or increase the response time to execute
        searches after the last key press, in increments of 100ms, where
        the default is 3 (300ms delay).  No whitespace may be given between
        -Q and its argument DELAY.  Initial patterns may be specified with
        -e PATTERN, i.e. a PATTERN argument requires option -e.  Press F1
        or CTRL-Z to view the help screen.  Press F2 or CTRL-Y to invoke a
        command to view or edit the file shown at the top of the screen.
        The command can be specified with option --view, or defaults to
        environment variable PAGER when defined, or EDITOR.  Press Tab and
        Shift-Tab to navigate directories and to select a file to search.
        Press Enter to select lines to output.  Press ALT-l for option -l
        to list files, ALT-n for -n, etc.  Non-option commands include
        ALT-] to increase context.  See also options --no-confirm, --delay,
        --split and --view.
--no-confirm
        Do not confirm actions in -Q query TUI.  The default is confirm.
--delay=DELAY
        Set the default -Q key response delay.  Default is 3 for 300ms.
--split
        Split the -Q query TUI screen on startup.
--view[=COMMAND]
        Use COMMAND to view/edit a file in -Q query TUI by pressing CTRL-Y.

This option starts a user interface to enter search patterns interactively:

  • Press F1 or CTRL-Z to view a help screen and to enable or disable options.
  • Press Alt with a key corresponding to a ugrep option letter or digit to enable or disable the ugrep option. For example, pressing Alt-c enables option -c to count matches. Pressing Alt-c again disables -c. Options can be toggled with the Alt key while searching or when viewing the help screen. If Alt/Meta keys are not working (e.g. X11 xterm), then press CTRL-O followed by the key corresponding to the option. Alt keys may work in xterm by adding xterm*metaSendsEscape: true to ~/.Xdefaults`.
  • Press Alt-g to enter or edit option -g file and directory matching globs, a comma-separated list of gitignore-style glob patterns. Presssing ESC returns control to the query pattern prompt (the globs are saved). When a glob is preceded by a ! or a ^, skips files whose name matches the glob When a glob contains a /, full pathnames are matched. Otherwise basenames are matched. When a glob ends with a /, directories are matched.
  • The query TUI prompt switches between Q> (normal), F> (fixed strings), G> (basic regex), P> (Perl matching), and Z> (fuzzy matching). When the --glob= prompt is shown, a comma-separated list of gitignore-style glob patterns may be entered. Presssing ESC returns control to the pattern prompt.
  • Press CTRL-T to split the TUI screen to preview a file in the bottom pane.
  • Press CTRL-Y to view a file with a pager specified with --view.
  • Press Enter to switch to selection mode to select lines to output when ugrep exits. Normally, ugrep in query mode does not output any results unless results are selected. While in selection mode, select or deselect lines with Enter or Del, or press A to select all results.
  • The file listed or shown at the top of the screen, or beneath the cursor in selection mode, is edited by pressing F2 or CTRL-Y. A file viewer or editor may be specified with --view=COMMAND. Otherwise, the PAGER or EDITOR environment variables are used to invoke the command with CTRL-Y. Filenames must be enabled and visible in the output to use this feature.
  • Press TAB to chdir one level down into the directory of the file listed or viewed at the top of the screen. If no directory exists, the file itself is selected to search. Press Shift-TAB to go back up one level.
  • Press CTRL-] to toggle colors on and off. Normally ugrep in query mode uses colors and other markup to highlight results. When colors are turned off, selected results are also not colored in the output produced by ugrep when ugrep exits. When colors are turned on (the default), selected results are colored depending on the --color option.
  • The query engine is optimized to limit system load by performing on-demand searches to produce results only for the visible parts shown in the interface. That is, results are shown on demand, when scrolling down and when exiting when all results are selected. When the search pattern is modified, the previous search query is cancelled when incomplete. This effectively limits the load on the system to maintain a high degree of responsiveness of the query engine to user input. Because the search results are produced on demand, occasionally you may notice a flashing "Searching..." message when searching files.
  • To display results faster, specify a low DELAY value such as 1. However, lower values may increase system load as a result of repeatedly initiating and cancelling searches by each key pressed.
  • To avoid long pathnames to obscure the view, --heading is enabled by default. Press Alt-+ to switch headings off.

Query TUI key mapping:

key(s) function
Alt-key toggle ugrep command-line option corresponding to key
Alt-/xxxx/ insert Unicode hex code point U+xxxx
Esc Ctrl-C go back or exit
Ctrl-Q quick exit and output the results selected in selection mode
Tab chdir to the directory of the file shown at the top of the screen or select file
Shift-Tab chdir one level up or deselect file
Enter enter selection mode and toggle selected lines to output on exit
Up Ctrl-P move up
Down Ctrl-N move down
Left Ctrl-B move left
Right Ctrl-F move right
PgUp Ctrl-G move display up by a page
PgDn Ctrl-D move display down by a page
Alt-Up move display up by 1/2 page (MacOS Shift-Up)
Alt-Down move display down by 1/2 page (MacOS Shift-Down)
Alt-Left move display left by 1/2 page (MacOS Shift-Left)
Alt-Right move display right by 1/2 page (MacOS Shift-Right)
Home Ctrl-A move cursor to the beginning of the line
End Ctrl-E move cursor to the end of the line
Ctrl-K delete after cursor
Ctrl-L refresh screen
Ctrl-O+key toggle ugrep command-line option corresponding to key, same as Alt-key
Ctrl-R F4 jump to bookmark
Ctrl-S jump to the next dir/file/context
Ctrl-T F5 toggle split screen (--split starts a split-screen TUI)
Ctrl-U delete before cursor
Ctrl-V verbatim character
Ctrl-W jump back one dir/file/context
Ctrl-X F3 set bookmark
Ctrl-Y F2 view or edit the file shown at the top of the screen
Ctrl-Z F1 view help and options
Ctrl-^ chdir back to the starting working directory
Ctrl-] toggle color/mono
Ctrl-\ terminate process

To interactively search the files in the working directory and below:

ug -Q

Same, but restricted to C++ files only and ignoring .gitignore files:

ug -Q -tc++ --ignore-files

To interactively search all makefiles in the working directory and below:

ug -Q -g 'Makefile*' -g 'makefile*'

Same, but for up to 2 directory levels (working and one subdirectory level):

ug -Q -2 -g 'Makefile*' -g 'makefile*'

To interactively view the contents of main.cpp and search it, where -y shows any nonmatching lines as context:

ug -Q -y main.cpp

To interactively search main.cpp, starting with the search pattern TODO and a match context of 5 lines (context can be interactively enabled and disabled, this also overrides the default context size of 2 lines):

ug -Q -C5 -e TODO main.cpp

To view and search the contents of an archive (e.g. zip, tarball):

ug -Q -z archive.tar.gz

To interactively select files from project.zip to decompress with unzip, using ugrep query selection mode (press Enter to select lines):

unzip project.zip `zipinfo -1 project.zip | ugrep -Q`

🔝 Back to table of contents

Recursively list matching files with -l, -R, -r, -S, --depth, -g, -O, and -t

 
-L, --files-without-match
        Only the names of files not containing selected lines are written
        to standard output.  Pathnames are listed once per file searched.
        If the standard input is searched, the string ``(standard input)''
        is written.
-l, --files-with-matches
        Only the names of files containing selected lines are written to
        standard output.  ugrep will only search a file until a match has
        been found, making searches potentially less expensive.  Pathnames
        are listed once per file searched.  If the standard input is
        searched, the string ``(standard input)'' is written.
-R, --dereference-recursive
        Recursively read all files under each directory.  Follow all
        symbolic links to files and directories, unlike -r.
-r, --recursive
        Recursively read all files under each directory, following symbolic
        links only if they are on the command line.  Note that when no FILE
        arguments are specified and input is read from a terminal,
        recursive searches are performed as if -r is specified.
-S, --dereference-files
        When -r is specified, symbolic links to files are followed, but not
        to directories.  The default is not to follow symbolic links.
--depth=[MIN,][MAX], -1, -2, -3, ... -9, -10, -11, -12, ...
        Restrict recursive searches from MIN to MAX directory levels deep,
        where -1 (--depth=1) searches the specified path without recursing
        into subdirectories.  Note that -3 -5, -3-5, and -35 search 3 to 5
        levels deep.  Enables -r if -R or -r is not specified.
-g GLOBS, --glob=GLOBS
        Search only files whose name matches the specified comma-separated
        list of GLOBS, same as --include='glob' for each `glob' in GLOBS.
        When a `glob' is preceded by a `!' or a `^', skip files whose name
        matches `glob', same as --exclude='glob'.  When `glob' contains a
        `/', full pathnames are matched.  Otherwise basenames are matched.
        When `glob' ends with a `/', directories are matched, same as
        --include-dir='glob' and --exclude-dir='glob'.  A leading `/'
        matches the working directory.  This option may be repeated and may
        be combined with options -M, -O and -t to expand searches.  See
        `ugrep --help globs' and `man ugrep' section GLOBBING for details.
-O EXTENSIONS, --file-extension=EXTENSIONS
        Search only files whose filename extensions match the specified
        comma-separated list of EXTENSIONS, same as --include='*.ext' for
        each `ext' in EXTENSIONS.  When `ext' is preceded by a `!' or a
        `^', skip files whose filename extensions matches `ext', same as
        --exclude='*.ext'.  This option may be repeated and may be combined
        with options -g, -M and -t to expand the recursive search.
-t TYPES, --file-type=TYPES
        Search only files associated with TYPES, a comma-separated list of
        file types.  Each file type corresponds to a set of filename
        extensions passed to option -O and filenames passed to option -g.
        For capitalized file types, the search is expanded to include files
        with matching file signature magic bytes, as if passed to option
        -M.  When a type is preceded by a `!' or a `^', excludes files of
        the specified type.  This option may be repeated.
--stats
        Output statistics on the number of files and directories searched,
        and the inclusion and exclusion constraints applied.

If no FILE arguments are specified and input is read from a terminal, recursive searches are performed as if -r is specified. To force reading from standard input, specify - as the FILE argument.

To recursively list all non-empty files in the working directory:

ug -r -l ''

To list all non-empty files in the working directory but not deeper (since a FILE argument is given, in this case . for the working directory):

ug -l '' .

To list all non-empty files in directory mydir but not deeper (since a FILE argument is given):

ug -l '' mydir

To list all non-empty files in directory mydir and deeper while following symlinks:

ug -R -l '' mydir

To recursively list all non-empty files on the path specified, while visiting subdirectories only, i.e. directories mydir/ and subdirectories at one level deeper mydir/*/ are visited (note that -2 -l can be abbreviated to -l2):

ug -2 -l '' mydir

To recursively list all non-empty files in directory mydir, not following any symbolic links (except when on the command line such as mydir):

ug -rl '' mydir

To recursively list all Makefiles matching the text CPP:

ug -l -tmake 'CPP'

To recursively list all Makefile.* matching bin_PROGRAMS:

ug -l -g'Makefile.*' 'bin_PROGRAMS'

To recursively list all non-empty files with extension .sh, with -Osh:

ug -l -Osh ''

To recursively list all shell scripts based on extensions and shebangs with -tShell:

ug -l -tShell ''

To recursively list all shell scripts based on extensions only with -tshell:

ug -l -tshell ''

🔝 Back to table of contents

Boolean query patterns with -%, -%%, --and, --not

 
--bool, -%, -%%
        Specifies Boolean query patterns.  A Boolean query pattern is
        composed of `AND', `OR', `NOT' operators and grouping with `(' `)'.
        Spacing between subpatterns is the same as `AND', `|' is the same
        as `OR' and a `-' is the same as `NOT'.  The `OR' operator binds
        more tightly than `AND'.  For example, --bool 'A|B C|D' matches
        lines with (`A' or `B') and (`C' or `D'), --bool 'A -B' matches
        lines with `A' and not `B'.  Operators `AND', `OR', `NOT' require
        proper spacing.  For example, --bool 'A OR B AND C OR D' matches
        lines with (`A' or `B') and (`C' or `D'), --bool 'A AND NOT B'
        matches lines with `A' without `B'.  Quoted subpatterns are matched
        literally as strings.  For example, --bool 'A "AND"|"OR"' matches
        lines with `A' and also either `AND' or `OR'.  Parentheses are used
        for grouping.  For example, --bool '(A B)|C' matches lines with `A'
        and `B', or lines with `C'.  Note that all subpatterns in a Boolean
        query pattern are regular expressions, unless -F is specified.
        Options -E, -F, -G, -P and -Z can be combined with --bool to match
        subpatterns as strings or regular expressions (-E is the default.)
        This option does not apply to -f FILE patterns.  The double short
        option -%% enables options --bool --files.  Option --stats displays
        the Boolean search patterns applied.  See also options --and,
        --andnot, --not, --files and --lines.
--files
        Boolean file matching mode, the opposite of --lines.  When combined
        with option --bool, matches a file if all Boolean conditions are
        satisfied.  For example, --bool --files 'A B|C -D' matches a file
        if some lines match `A', and some lines match either `B' or `C',
        and no line matches `D'.  See also options --and, --andnot, --not,
        --bool and --lines.  The double short option -%% enables options
        --bool --files.
--lines
        Boolean line matching mode for option --bool, the default mode.
--and [[-e] PATTERN] ... -e PATTERN
        Specify additional patterns to match.  Patterns must be specified
        with -e.  Each -e PATTERN following this option is considered an
        alternative pattern to match, i.e. each -e is interpreted as an OR
        pattern.  For example, -e A -e B --and -e C -e D matches lines with
        (`A' or `B') and (`C' or `D').  Note that multiple -e PATTERN are
        alternations that bind more tightly together than --and.  Option
        --stats displays the search patterns applied.  See also options
        --not, --andnot, and --bool.
--andnot [[-e] PATTERN] ...
        Combines --and --not.  See also options --and, --not, and --bool.
--not [-e] PATTERN
        Specifies that PATTERN should not match.  Note that -e A --not -e B
        matches lines with `A' or lines without a `B'.  To match lines with
        `A' that have no `B', specify -e A --andnot -e B.  Option --stats
        displays the search patterns applied.  See also options --and,
        --andnot, and --bool.
--stats
        Output statistics on the number of files and directories searched,
        and the inclusion and exclusion constraints applied.

Note that the --and, --not, and --andnot options require -e PATTERN.

The -% option makes all patterns Boolean-based, supporting the following logical operations listed from the highest level of precedence to the lowest:

operator alternative result
"x" match x literally and exactly as specified (using the standard regex escapes \Q and \E)
( ) Boolean expression grouping
-x NOT x inverted match, i.e. matches if x does not match
x|y x OR y matches lines with x or y
x y x AND y matches lines with both x and y

  • x and y are subpatterns that do not start with the special symbols |, -, and ( (use quotes or a \ escape to match these);

  • - and NOT are the same and take precedence over OR, which means that -x|y == (-x)|y for example.

  • | and OR are the same and take precedence over AND, which means that x y|z == x (y|z) for example;

The --stats option displays the Boolean queries in human-readable form converted to CNF (Conjunctive Normal Form), after the search is completed. To show the CNF without a search, read from standard input terminated by an EOF, like echo | ugrep -% '...' --stats.

Subpatterns are color-highlighted in the output, except those negated with NOT (a NOT subpattern may still show up in a matching line when using an OR-NOT pattern like x|-y). Note that subpatterns may overlap. In that case only the first matching subpattern is color-highlighted.

Multiple lines may be matched when subpatterns match newlines. There is one exception however: subpatterns ending with (?=X) lookaheads may not match when X spans multiple lines.

Empty patterns match any line (grep standard). Therefore, -% 'x|""|y' matches everything and x and y are not color-highlighted. Option -y should be used to show every line as context, for example -y 'x|y'.

Fzf-like interactive querying (Boolean search with fixed strings with fuzzy matching to allow e.g. up to 4 extra characters matched with -Z+4 in words with -w), press TAB and ALT-y to view a file with matches. Press SHIFT-TAB and ALT-l to go back to the list of matching files:

ug -Q -%% -l -w -F -Z+4 --sort=best

To recursively find all files containing both hot and dog anywhere in the file with option --files:

ug -%% 'hot dog'
ug --files -e hot --and dog

To find lines containing both hot and dog in myfile.txt:

ug -% 'hot dog' myfile.txt
ug -e hot --and dog myfile.txt

To find lines containing place and then also hotdog or taco (or both) in myfile.txt:

ug -% 'hotdog|taco place' myfile.txt
ug -e hotdog -e taco --and place myfile.txt

Same, but exclude lines matching diner:

ug -% 'hotdog|taco place -diner' myfile.txt
ug -e hotdog -e taco --and place --andnot diner myfile.txt

To find lines with diner or lines that match both fast and food but not bad in myfile.txt:

ug -% 'diner|(fast food -bad)' myfile.txt

To find lines with fast food (exactly) or lines with diner but not bad or old in myfile.txt:

ug -% '"fast food"|diner -bad -old' myfile.txt

Same, but using a different Boolean expression that has the same meaning:

ug -% '"fast food"|diner -(bad|old)' myfile.txt

To find lines with diner implying good in myfile.txt (that is, show lines with good without diner and show lines with diner but only those with good, which is logically implied!):

ug -% 'good|-diner' myfile.txt
ug -e good --not diner myfile.txt

To find lines with foo and -bar and "baz" in myfile.txt (not that - and " should be matched using \ escapes and with --and -e -bar):

ug -% 'foo \-bar \"baz\"' myfile.txt
ug -e foo --and -e -bar --and '"baz"' myfile.txt

To search myfile.cpp for lines with TODO or FIXME but not both on the same line, like XOR:

ug -% 'TODO|FIXME -(TODO FIXME)' myfile.cpp
ug -e TODO -e FIXME --and --not TODO --not FIXME myfile.cpp

🔝 Back to table of contents

Search this but not that with -v, -e, -N, -f, -L, -w, -x

 
-e PATTERN, --regexp=PATTERN
        Specify a PATTERN to search the input.  An input line is selected
        if it matches any of the specified patterns.  This option is useful
        when multiple -e options are used to specify multiple patterns, or
        when a pattern begins with a dash (`-'), or to specify a pattern
        after option -f or after the FILE arguments.
-f FILE, --file=FILE
        Read newline-separated patterns from FILE.  White space in patterns
        is significant.  Empty lines in FILE are ignored.  If FILE does not
        exist, the GREP_PATH environment variable is used as path to FILE.
        If that fails, looks for FILE in /usr/local/share/ugrep/pattern.
        When FILE is a `-', standard input is read.  This option may be
        repeated.
-L, --files-without-match
        Only the names of files not containing selected lines are written
        to standard output.  Pathnames are listed once per file searched.
        If the standard input is searched, the string ``(standard input)''
        is written.
-N PATTERN, --neg-regexp=PATTERN
        Specify a negative PATTERN to reject specific -e PATTERN matches
        with a counter pattern.  Note that longer patterns take precedence
        over shorter patterns, i.e. a negative pattern must be of the same
        length or longer to reject matching patterns.  Option -N cannot be
        specified with -P.  This option may be repeated.
-v, --invert-match
        Selected lines are those not matching any of the specified
        patterns.
-w, --word-regexp
        The PATTERN is searched for as a word, such that the matching text
        is preceded by a non-word character and is followed by a non-word
        character.  Word-like characters are Unicode letters, digits and
        connector punctuations such as underscore.
-x, --line-regexp
        Select only those matches that exactly match the whole line, as if
        the patterns are surrounded by ^ and $.

See also Boolean query patterns with -%, -%%, --and, --not for more powerful Boolean query options than the traditional GNU/BSD grep options.

To display lines in file myfile.sh but not lines matching ^[ \t]*#:

ug -v '^[ \t]*#' myfile.sh

To search myfile.cpp for lines with FIXME and urgent, but not Scotty:

ugrep FIXME myfile.cpp | ugrep urgent | ugrep -v Scotty

Same, but using -% for Boolean queries:

ug -% 'FIXME urgent -Scotty' myfile.cpp

To search for decimals using pattern \d+ that do not start with 0 using negative pattern 0\d+ and excluding 555:

ug -e '\d+' -N '0\d+' -N 555 myfile.cpp

To search for words starting with disp without matching display in file myfile.py by using a "negative pattern" -N '/<display\>' where -N specifies an additional negative pattern to skip matches:

ug -e '\<disp' -N '\<display\>' myfile.py

To search for lines with the word display in file myfile.py skipping this word in strings and comments, where -f specifies patterns in files which are predefined patterns in this case:

ug -n -w 'display' -f python/zap_strings -f python/zap_comments myfile.py

To display lines that are not blank lines:

ug -x -e '.*' -N '\h*' myfile.py

Same, but using -v and -x with \h*, i.e. pattern ^\h*$:

ug -v -x '\h*' myfile.py

To recursively list all Python files that do not contain the word display, allowing the word to occur in strings and comments:

ug -RL -tPython -w 'display' -f python/zap_strings -f python/zap_comments

🔝 Back to table of contents

Search non-Unicode files with --encoding

 
--encoding=ENCODING
        The encoding format of the input.  The default ENCODING is binary
        and UTF-8 which are the same.  Note that option -U specifies binary
        PATTERN matching (text matching is the default.)

Binary, ASCII and UTF-8 files do not require this option to search them. Also UTF-16 and UTF-32 files do not require this option to search them, assuming that UTF-16 and UTF-32 files start with a UTF BOM (byte order mark) as usual. Other file encodings require option --encoding=ENCODING:

encoding parameter
ASCII n/a
UTF-8 n/a
UTF-16 with BOM n/a
UTF-32 with BOM n/a
UTF-16 BE w/o BOM UTF-16 or UTF-16BE
UTF-16 LE w/o BOM UTF-16LE
UTF-32 w/o BOM UTF-32 or UTF-32BE
UTF-32 w/o BOM UTF-32LE
Latin-1 LATIN1 or ISO-8859-1
ISO-8859-1 ISO-8859-1
ISO-8859-2 ISO-8859-2
ISO-8859-3 ISO-8859-3
ISO-8859-4 ISO-8859-4
ISO-8859-5 ISO-8859-5
ISO-8859-6 ISO-8859-6
ISO-8859-7 ISO-8859-7
ISO-8859-8 ISO-8859-8
ISO-8859-9 ISO-8859-9
ISO-8859-10 ISO-8859-10
ISO-8859-11 ISO-8859-11
ISO-8859-13 ISO-8859-13
ISO-8859-14 ISO-8859-14
ISO-8859-15 ISO-8859-15
ISO-8859-16 ISO-8859-16
MAC (CR=newline) MAC
MacRoman (CR=newline) MACROMAN
EBCDIC EBCDIC
DOS code page 437 CP437
DOS code page 850 CP850
DOS code page 858 CP858
Windows code page 1250 CP1250
Windows code page 1251 CP1251
Windows code page 1252 CP1252
Windows code page 1253 CP1253
Windows code page 1254 CP1254
Windows code page 1255 CP1255
Windows code page 1256 CP1256
Windows code page 1257 CP1257
Windows code page 1258 CP1258
KOI8-R KOI8-R
KOI8-U KOI8-U
KOI8-RU KOI8-RU

Note that regex patterns are always specified in UTF-8 (includes ASCII). To search binary files with binary patterns, see searching and displaying binary files with -U, -W, and -X.

To recursively list all files that are ASCII (i.e. 7-bit):

ug -L '[^[:ascii:]]'

To recursively list all files that are non-ASCII, i.e. UTF-8, UTF-16, and UTF-32 files with non-ASCII Unicode characters (U+0080 and up):

ug -l '[^[:ascii:]]'

To check if a file contains non-ASCII Unicode (U+0080 and up):

ug -q '[^[:ascii:]]' myfile && echo "contains Unicode"

To remove invalid Unicode characters from a file (note that -o may not work because binary data is detected and rejected and newlines are added, but --format="%o% does not check for binary and copies the match "as is"):

ug "[\p{Unicode}\n]" --format="%o" badfile.txt

To recursively list files with invalid UTF content (i.e. invalid UTF-8 byte sequences or files that contain any UTF-8/16/32 code points that are outside the valid Unicode range) by matching any code point with . and by using a negative pattern -N '\p{Unicode}' to ignore each valid Unicode character:

ug -l -e '.' -N '\p{Unicode}'

To display lines containing laughing face emojis:

ug '[😀-😏]' emojis.txt

The same results are obtained using \x{hhhh} to select a Unicode character range:

ug '[\x{1F600}-\x{1F60F}]' emojis.txt

To display lines containing the names Gödel (or Goedel), Escher, or Bach:

ug 'G(ö|oe)del|Escher|Bach' GEB.txt wiki.txt

To search for lorem in lower or upper case in a UTF-16 file that is marked with a UTF-16 BOM:

ug -iw 'lorem' utf16lorem.txt

To search utf16lorem.txt when this file has no UTF-16 BOM, using --encoding:

ug --encoding=UTF-16 -iw 'lorem' utf16lorem.txt

To search file spanish-iso.txt encoded in ISO-8859-1:

ug --encoding=ISO-8859-1 -w 'año' spanish-iso.txt

🔝 Back to table of contents

Matching multiple lines of text

 
-o, --only-matching
        Output only the matching part of lines.  If -A, -B or -C is
        specified, fits the match and its context on a line within the
        specified number of columns.

Multiple lines may be matched by patterns that match newline characters. Use option -o to output the match only, not the full lines(s) that match.

To match a \n line break, include \n in the pattern to match the LF character. If you want to match \r\n and \n line breaks, use \r?\n or simply use \R to match any Unicode line break \r\n, \r, \v, \f, \n, U+0085, U+2028 and U+2029.

To match C/C++ /*...*/ multi-line comments:

ug '/\*(.*\n)*?.*\*+\/' myfile.cpp

To match C/C++ comments using the predefined c/comments patterns with -f c/comments, restricted to the matching part only with option -o:

ug -of c/comments myfile.cpp

Same as sed -n '/begin/,/end/p': to match all lines between a line containing begin and the first line after that containing end, using lazy repetition:

ug -o '.*begin(.|\n)*?end.*' myfile.txt

🔝 Back to table of contents

Displaying match context with -A, -B, -C, -y, and --width

more like this

gameboycss

A Visual Nintendo GameBoy using HTML, CSS, Javascript & WebComponents 👾 (old version)

JavaScript★ 56
@ManzDev

search

search projects, people, and tags