Vim Tips Wiki
No edit summary
Tag: sourceedit
Tag: sourceedit
(12 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
This page explains several strategies for dealing with long lines. Vim does not handle long lines well; there are restrictions on when a long line can partially be displayed, and commands like <code>&lt;C-E&gt;</code> act on logical lines rather than visual lines. Part of the reason for this is that Vim is a descendant of [[Wikipedia:Line editor|line-based editors]] that only acted on logical lines. The upshot is that Vim will often feel "jumpy" when scrolling and moving in a buffer with long lines, especially ones that fill up the entire window.
 
This page explains several strategies for dealing with long lines. Vim does not handle long lines well; there are restrictions on when a long line can partially be displayed, and commands like <code>&lt;C-E&gt;</code> act on logical lines rather than visual lines. Part of the reason for this is that Vim is a descendant of [[Wikipedia:Line editor|line-based editors]] that only acted on logical lines. The upshot is that Vim will often feel "jumpy" when scrolling and moving in a buffer with long lines, especially ones that fill up the entire window.
   
You may encounter files with long lines in various situations:
+
One may encounter files with long lines in various situations:
   
* Conventions can require it; e.g. the English Wikipedia favors single-line paragraphs so lines tend to be long
+
* Conventions can require it; e.g. the English Wikipedia, which uses the MediaWiki markup language, favors single-line paragraphs so lines tend to be long
 
* Some data files (JSON, CSVs, etc.) use long lines
 
* Some data files (JSON, CSVs, etc.) use long lines
 
* Content on the web is often presented in [[Wikipedia:Minification (programming)|minified]] form, leading to long lines
 
* Content on the web is often presented in [[Wikipedia:Minification (programming)|minified]] form, leading to long lines
Line 10: Line 10:
   
 
* Making use of Vim's built-in capabilities for navigating long lines (i.e. work around long lines without modifying them)
 
* Making use of Vim's built-in capabilities for navigating long lines (i.e. work around long lines without modifying them)
* Preprocessing the file to use shorter lines; the two strategies here are "chopping" and pretty-printing
+
* Preprocessing the file to use shorter lines, and then possibly converting back to using long lines after the editing task in Vim is done; the two strategies here are "chopping" and pretty-printing
* Increasing the terminal window size so that more text can fit in
+
* Increasing the terminal window size or decreasing the terminal font size so that more text can fit in
 
* Taking steps to prevent long lines (rewriting code to use shorter lines, enforcing a manageable text width, etc.)
 
* Taking steps to prevent long lines (rewriting code to use shorter lines, enforcing a manageable text width, etc.)
* Giving up on Vim for these files and using editors that work fine with long lines (gedit, GNU Emacs, etc.)
+
* Giving up on Vim for these files and using editors and pagers that work fine with long lines (less, gedit, GNU Emacs, etc.); it's important to keep in mind that there is no single best tool for every task
   
 
This article will cover the first two points.
 
This article will cover the first two points.
Line 19: Line 19:
 
==Navigating long lines with Vim's built-in capabilities==
 
==Navigating long lines with Vim's built-in capabilities==
   
Vim can attempt to show long lines with <code>:set wrap</code> (on by default). With <code>:set linebreak</code>, Vim will wrap the lines at characters in <code>'breakat'</code>; this generally makes the text easier to read. Also very useful is <code>:set display+=lastline</code>, which will try to show as much as possible of the last line in the window (rather than a column of "@", which is the default behavior).
+
Vim can attempt to show long lines with <code>:set wrap</code> (on by default). With <code>:set linebreak</code>, Vim will wrap the lines at characters in <code>'breakat'</code>; this generally makes the text easier to read (in older versions of Vim, one will also have to <code>:set nolist</code>). Also very useful is <code>:set display+=lastline</code>, which will try to show as much as possible of the last line in the window (rather than a column of "@", which is the default behavior).
   
 
The normal mode movements <code>j</code> and <code>k</code> move by logical lines, but these have the analogous <code>gj</code> and <code>gk</code> for moving by visual lines. Similarly, <code>g0</code>, <code>g^</code>, and <code>g$</code> all work. In addition, <code>gm</code> will move the cursor to the middle of the visual line.
 
The normal mode movements <code>j</code> and <code>k</code> move by logical lines, but these have the analogous <code>gj</code> and <code>gk</code> for moving by visual lines. Similarly, <code>g0</code>, <code>g^</code>, and <code>g$</code> all work. In addition, <code>gm</code> will move the cursor to the middle of the visual line.
   
Search-to-navigate is another useful way to move. It's advisable to <code>:set incsearch</code> so that you can see the cursor move as you type, and then hit enter when you get to the desired location.
+
Search-to-navigate is another useful way to move. It's advisable to <code>:set incsearch</code> so that one can see the cursor move as one types, and then hit enter when one gets to the desired location.
   
 
Note that the commands <code>f</code>, <code>t</code>, <code>F</code>, <code>T</code>, <code>;</code>, and <code>,</code> are even more powerful within long lines: they can cover more ground.
 
Note that the commands <code>f</code>, <code>t</code>, <code>F</code>, <code>T</code>, <code>;</code>, and <code>,</code> are even more powerful within long lines: they can cover more ground.
   
  +
If the buffer contains prose, the sentence motions <code>)</code> and <code>(</code> also work well.
Using <code>:set scrolloff=99999</code> (or any large number instead of 99999) will try to keep the cursor in the middle of the screen, providing more context even within very long lines. This will allow you to type something like <code>5gj</code> to simulate <code>5&lt;C-E&gt;</code>.
 
  +
 
Using <code>:set scrolloff=99999</code> (or any large number instead of 99999) will try to keep the cursor in the middle of the screen, providing more context even within very long lines. This will allow one to type something like <code>5gj</code> to simulate <code>5&lt;C-E&gt;</code>.
  +
  +
Note that when using commands like <code>:vimgrep</code> and <code>:clist</code>, Vim uses a <code>more</code>-like pager that can display partial lines even when using <code>j</code> and <code>k</code>.
  +
  +
The following provides the maps <code>gH</code>, <code>gM</code>, and <code>gL</code> to emulate <code>H</code>, <code>M</code>, and <code>L</code> even when very long lines are present. Note that Vim already defines the normal mode command <code>gH</code> ({{help|gH}}), although this is to enter Select mode, which is rarely used.
  +
 
<pre>
  +
nnoremap &lt;silent&gt; &lt;expr&gt; gH winline() - 1 - &amp;scrolloff &gt; 0
  +
\ ? ':normal! ' . (winline() - 1 - &amp;scrolloff) . 'gkg^&lt;CR&gt;'
  +
\ : 'g^'
  +
nnoremap &lt;silent&gt; &lt;expr&gt; gM winline() &lt; (winheight(0)+1)/2
  +
\ ? ':normal! ' . ((winheight(0)+1)/2 - winline()) . 'gjg^&lt;CR&gt;'
  +
\ : winline() == (winheight(0)+1)/2
  +
\ ? 'g^'
  +
\ : ':normal! ' . (winline() - (winheight(0)+1)/2) . 'gkg^&lt;CR&gt;'
  +
nnoremap &lt;silent&gt; &lt;expr&gt; gL winheight(0) - winline() - &amp;scrolloff &gt; 0
  +
\ ? ':normal! ' . (winheight(0) - winline() - &amp;scrolloff) . 'gjg^&lt;CR&gt;'
  +
\ : 'g^'
 
</pre>
   
 
==Preprocessing long lines==
 
==Preprocessing long lines==
Line 33: Line 53:
 
===Generic formatting tools===
 
===Generic formatting tools===
   
Most UNIX-like programming environments offer generic tools for formatting text. These include <code>fmt</code>, <code>fold</code>, and <code>par</code>. From inside Vim, one can invoke:
+
Most UNIX-like programming environments offer generic tools for formatting text. These include <code>fmt</code>, <code>fold</code>, <code>sed</code>, <code>perl</code>, and <code>par</code>. From inside Vim, one can invoke:
   
 
<pre>
 
<pre>
Line 43: Line 63:
 
<code>fold</code> is more aggressive: it will chop lines even inside of words.
 
<code>fold</code> is more aggressive: it will chop lines even inside of words.
   
In addition, Vim itself provides the command <code>gqq</code> to format the current line, so something like:
+
In addition, Vim itself provides the command <code>gqq</code> to format the current line, so typing something like <code>999gqq</code> (with maybe hitting <code>.</code> a few times if that's not enough to format the whole buffer) in normal mode will format the buffer according to the value of <code>'textwidth'</code>.
   
 
If the original linebreak locations must be preserved, one can leave a unique character in place of linebreaks. Below, the character '↵' (typed using <code>&lt;C-V&gt;u21B5</code>) is provided as a suggestion, but it must not appear in the file itself.
<pre>
 
:%normal! gqq
 
</pre>
 
 
will format the buffer according to the value of <code>'textwidth'</code>.
 
 
If the original linebreak locations must be preserved, you can leave a unique character in place of linebreaks. Below, the character '↵' (typed using <code>&lt;C-V&gt;u21B5</code>) is provided as a suggestion, but it must not appear in the file itself.
 
   
 
<pre>
 
<pre>
  +
" Use :ShortLines or :ShortLinesAtSpace to convert the document to use
  +
" short lines with hard linebreaks, and use :LongLines to convert back.
 
command! ShortLines :%s/.\{71}/&amp;↵\r/g | 1
 
command! ShortLines :%s/.\{71}/&amp;↵\r/g | 1
 
command! ShortLinesAtSpace :%s/.\{70,79} /&amp;↵\r/g | 1
 
command! ShortLinesAtSpace :%s/.\{70,79} /&amp;↵\r/g | 1
Line 65: Line 81:
 
Instead of generic tools, one can sometimes do better by using a tool suited to a particular filetype.
 
Instead of generic tools, one can sometimes do better by using a tool suited to a particular filetype.
   
If you are working with JSON files and have Python installed (note: this is just the system binary rather than Python support being compiled into Vim):
+
If one is working with JSON files and have Python installed (note: this is just the system binary rather than Python support being compiled into Vim):
   
 
<pre>
 
<pre>
Line 73: Line 89:
 
will convert the buffer into pretty-printed JSON. There are also plugins like [https://github.com/tpope/vim-jdaddy jdaddy.vim] that are designed to make work with JSON easier.
 
will convert the buffer into pretty-printed JSON. There are also plugins like [https://github.com/tpope/vim-jdaddy jdaddy.vim] that are designed to make work with JSON easier.
   
For HTML and XML, there are tools like <code>tidy</code> and <code>xmllint</code>; see [[Cleanup your HTML]] for the main article.
+
For HTML and XML, there are tools like <code>tidy</code> (e.g. <code>:%!tidy -utf8 -i</code>) and <code>xmllint</code>; see [[Cleanup your HTML]] for the main article.
   
 
In some cases, <code>gg=G</code> might make things nicer.
 
In some cases, <code>gg=G</code> might make things nicer.
Line 86: Line 102:
 
==See also==
 
==See also==
   
* [[Word wrap without line breaks]]
 
 
* {{help|25.4}}, "Dealing with long lines", which has a couple of parts:
 
* {{help|25.4}}, "Dealing with long lines", which has a couple of parts:
 
** {{help|edit-no-break}}
 
** {{help|edit-no-break}}
  +
** [https://vimhelp.appspot.com/usr_25.txt.html#edit-paragraph-join :help edit-paragraph-join]; this is similar to [[Copy paragraphs without excess newlines to MS Word]]
** {{help|edit-paragraph-join}} (the version of the help files on vimdoc seems to be old so this won't work; just type it in Vim)
 
 
* [[Word wrap without line breaks]]
  +
* [[Chop long lines]]
  +
* [[Format only long lines]]
  +
* [[Move through wrapped lines]]
  +
* [[Move cursor by display lines when wrapping]]
   
==References==
+
==External links==
   
* [https://github.com/neovim/neovim/issues/4232 How can Neovim implement scrolling based on screen-lines in long strings of wrapped text?] an issue filed in the Neovim development issues tracker; this might be the only real shot at getting better long lines support in Vim/Neovim.
+
* [https://github.com/neovim/neovim/issues/4232 "How can Neovim implement scrolling based on screen-lines in long strings of wrapped text?"], an issue filed in the Neovim development issues tracker; this might be the only real shot at getting better long lines support in Vim/Neovim.
 
* [http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf "Structural Regular Expressions"], an essay by Rob Pike on breaking out of the idea of line-based regular expressions.
 
* [http://doc.cat-v.org/bell_labs/structural_regexps/se.pdf "Structural Regular Expressions"], an essay by Rob Pike on breaking out of the idea of line-based regular expressions.

Revision as of 07:27, 8 February 2017

This page explains several strategies for dealing with long lines. Vim does not handle long lines well; there are restrictions on when a long line can partially be displayed, and commands like <C-E> act on logical lines rather than visual lines. Part of the reason for this is that Vim is a descendant of line-based editors that only acted on logical lines. The upshot is that Vim will often feel "jumpy" when scrolling and moving in a buffer with long lines, especially ones that fill up the entire window.

One may encounter files with long lines in various situations:

  • Conventions can require it; e.g. the English Wikipedia, which uses the MediaWiki markup language, favors single-line paragraphs so lines tend to be long
  • Some data files (JSON, CSVs, etc.) use long lines
  • Content on the web is often presented in minified form, leading to long lines

At a high level, the strategies for dealing with long lines can be grouped in the following way:

  • Making use of Vim's built-in capabilities for navigating long lines (i.e. work around long lines without modifying them)
  • Preprocessing the file to use shorter lines, and then possibly converting back to using long lines after the editing task in Vim is done; the two strategies here are "chopping" and pretty-printing
  • Increasing the terminal window size or decreasing the terminal font size so that more text can fit in
  • Taking steps to prevent long lines (rewriting code to use shorter lines, enforcing a manageable text width, etc.)
  • Giving up on Vim for these files and using editors and pagers that work fine with long lines (less, gedit, GNU Emacs, etc.); it's important to keep in mind that there is no single best tool for every task

This article will cover the first two points.

Navigating long lines with Vim's built-in capabilities

Vim can attempt to show long lines with :set wrap (on by default). With :set linebreak, Vim will wrap the lines at characters in 'breakat'; this generally makes the text easier to read (in older versions of Vim, one will also have to :set nolist). Also very useful is :set display+=lastline, which will try to show as much as possible of the last line in the window (rather than a column of "@", which is the default behavior).

The normal mode movements j and k move by logical lines, but these have the analogous gj and gk for moving by visual lines. Similarly, g0, g^, and g$ all work. In addition, gm will move the cursor to the middle of the visual line.

Search-to-navigate is another useful way to move. It's advisable to :set incsearch so that one can see the cursor move as one types, and then hit enter when one gets to the desired location.

Note that the commands f, t, F, T, ;, and , are even more powerful within long lines: they can cover more ground.

If the buffer contains prose, the sentence motions ) and ( also work well.

Using :set scrolloff=99999 (or any large number instead of 99999) will try to keep the cursor in the middle of the screen, providing more context even within very long lines. This will allow one to type something like 5gj to simulate 5<C-E>.

Note that when using commands like :vimgrep and :clist, Vim uses a more-like pager that can display partial lines even when using j and k.

The following provides the maps gH, gM, and gL to emulate H, M, and L even when very long lines are present. Note that Vim already defines the normal mode command gH (:help gH), although this is to enter Select mode, which is rarely used.

nnoremap <silent> <expr> gH winline() - 1 - &scrolloff > 0
      \ ? ':normal! ' . (winline() - 1 - &scrolloff) . 'gkg^<CR>'
      \ : 'g^'
nnoremap <silent> <expr> gM winline() < (winheight(0)+1)/2
      \ ? ':normal! ' . ((winheight(0)+1)/2 - winline()) . 'gjg^<CR>'
      \ : winline() == (winheight(0)+1)/2
      \         ? 'g^'
      \         : ':normal! ' . (winline() - (winheight(0)+1)/2) . 'gkg^<CR>'
nnoremap <silent> <expr> gL winheight(0) - winline() - &scrolloff > 0
      \ ? ':normal! ' . (winheight(0) - winline() - &scrolloff) . 'gjg^<CR>'
      \ : 'g^'

Preprocessing long lines

Generic formatting tools

Most UNIX-like programming environments offer generic tools for formatting text. These include fmt, fold, sed, perl, and par. From inside Vim, one can invoke:

:%!fmt -80 -s

to format the current buffer. The -80 specifies 80 columns as the desired width and the -s tells fmt to split lines only (i.e. to not join short lines together, called "refilling").

fold is more aggressive: it will chop lines even inside of words.

In addition, Vim itself provides the command gqq to format the current line, so typing something like 999gqq (with maybe hitting . a few times if that's not enough to format the whole buffer) in normal mode will format the buffer according to the value of 'textwidth'.

If the original linebreak locations must be preserved, one can leave a unique character in place of linebreaks. Below, the character '↵' (typed using <C-V>u21B5) is provided as a suggestion, but it must not appear in the file itself.

" Use :ShortLines or :ShortLinesAtSpace to convert the document to use
" short lines with hard linebreaks, and use :LongLines to convert back.
command! ShortLines :%s/.\{71}/&↵\r/g | 1
command! ShortLinesAtSpace :%s/.\{70,79} /&↵\r/g | 1
command! LongLines :%s/↵\n// | 1

Note that this sort of regex replacement will be slow on large files.

Filetype-specific tools

Instead of generic tools, one can sometimes do better by using a tool suited to a particular filetype.

If one is working with JSON files and have Python installed (note: this is just the system binary rather than Python support being compiled into Vim):

:%!python -m json.tool

will convert the buffer into pretty-printed JSON. There are also plugins like jdaddy.vim that are designed to make work with JSON easier.

For HTML and XML, there are tools like tidy (e.g. :%!tidy -utf8 -i) and xmllint; see Cleanup your HTML for the main article.

In some cases, gg=G might make things nicer.

Some plugins that try to unify this sort of pretty-printing/tidying are:

Languages like Go have strong source formatting requirements and provide tools like gofmt.

See also

External links