You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: repositories/neovim/issues/1008.json
+74-15Lines changed: 74 additions & 15 deletions
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,7 @@
3
3
"assignee": null,
4
4
"assignees": [],
5
5
"author_association": "MEMBER",
6
-
"body": "Followup of https://github.com/neovim/neovim/pull/978#issuecomment-49671122 . \n\nIn Neovim we are mostly using pipes instead of temp files for reliability and [performance](https://github.com/neovim/neovim/pull/978#issuecomment-50092527).\n\nBut there's an edge that mentioned in Vim `:help 'shelltemp'`:\n\n```\n The advantage of using a temp file is that the file type and encoding\n can be detected.\n```\n\nWe need to verify that this edge is not present in Neovim, or find a workaround, or update the documentation. [A vim-sensible issue](https://github.com/tpope/vim-sensible/issues/73) gives a couple examples of plugins that seem to be affected by this edge case.\n\nVim source and [unanswered question on vim_dev](https://groups.google.com/forum/#!msg/vim_dev/EHMfNBsdvrg/vVth0Wv90QcJ) do not give any more explanation about the edge case.\n\nHowever [this SO post](http://stackoverflow.com/a/17177904/152142) gives insight into the behavior of encodings on Windows.\n\n> if a string is not in the UTF-16LE encoding to begin with, a Win32 program must pass the correct codepage to MultiByteToWideChar. Also, WriteConsoleW will not work if the program’s output is redirected; more fiddling is needed in that case.\n\nThat post and the vim-sensible issue report suggest that this issue may be Windows-only, and possibly specific to cmd.exe. \n\nNext steps:\n- reproduce edge case in Vim, then try the same steps in Neovim\n- these plugins were [reported](https://github.com/tpope/vim-sensible/issues/73#issuecomment-49541650) to mess up with `set noshelltemp` and [`chcp 850`](https://github.com/tpope/vim-sensible/issues/73#issuecomment-50536097):\n - https://github.com/mileszs/ack.vim\n - https://github.com/dyng/ctrlsf.vim\n- is it Windows-only?\n - if so, is it specific to cmd.exe?\n - if so, perhaps we can avoid the issue entirely by using syscalls instead of building shell commands (@aktau's work achieves this [except for `do_filter`](https://github.com/neovim/neovim/pull/978#issuecomment-49699155)).\n\n---\n\nThe more I stare at the help text, the more I think this is related to Vim autocmds rather than some mysterious OS behavior. \n- Detecting the file type is _almost certainly_ a reference the `ftdetect` Vim mechanism.\n- Detecting the encoding _could_ refer to some feature of Vim that tries to infer the encoding based on the BOM or the presence of UTF8 sequences.\n\n---\n\nNotes on Windows:\n\nhttp://blogs.msdn.com/b/oldnewthing/archive/2005/03/08/389527.aspx\n- there are typically two 8-bit code pages active, the so-called \"ANSI\" code page and the so-called \"OEM\" code page\n- cmd.exe `/U` forces Unicode for piped/redirected output\n",
6
+
"body": "Followup of https://github.com/neovim/neovim/pull/978#issuecomment-49671122 . \n\nIn Neovim we are mostly using pipes instead of temp files for reliability and [performance](https://github.com/neovim/neovim/pull/978#issuecomment-50092527).\n\nBut there's an edge that mentioned in Vim `:help 'shelltemp'`:\n\n```\n The advantage of using a temp file is that the file type and encoding\n can be detected.\n```\n\nWe need to verify that this edge is not present in Neovim, or find a workaround, or update the documentation. [A vim-sensible issue](https://github.com/tpope/vim-sensible/issues/73) gives a couple examples of plugins that seem to be affected by this edge case.\n\nVim source and [unanswered question on vim_dev](https://groups.google.com/forum/#!msg/vim_dev/EHMfNBsdvrg/vVth0Wv90QcJ) do not give any more explanation about the edge case.\n\nHowever [this SO post](http://stackoverflow.com/a/17177904/152142) gives insight into the behavior of encodings on Windows.\n\n> if a string is not in the UTF-16LE encoding to begin with, a Win32 program must pass the correct codepage to MultiByteToWideChar. Also, WriteConsoleW will not work if the program’s output is redirected; more fiddling is needed in that case.\n\nThat post and the vim-sensible issue report suggest that this issue may be Windows-only, and possibly specific to cmd.exe. \n\nNext steps:\n- reproduce edge case in Vim, then try the same steps in Neovim\n- these plugins were [reported](https://github.com/tpope/vim-sensible/issues/73#issuecomment-49541650) to mess up with `set noshelltemp` and [`chcp 850`](https://github.com/tpope/vim-sensible/issues/73#issuecomment-50536097):\n - https://github.com/mileszs/ack.vim\n - https://github.com/dyng/ctrlsf.vim\n- is it Windows-only?\n - if so, is it specific to cmd.exe?\n - if so, perhaps we can avoid the issue entirely by using syscalls instead of building shell commands (@aktau's work achieves this [except for `do_filter`](https://github.com/neovim/neovim/pull/978#issuecomment-49699155)).\n\n---\n\nThe more I stare at the help text, the more I think this is related to Vim autocmds rather than some mysterious OS behavior. \n- Detecting the file type is _almost certainly_ a reference the `ftdetect` Vim mechanism.\n- Detecting the encoding _could_ refer to some feature of Vim that tries to infer the encoding based on the BOM or the presence of UTF8 sequences.\n\n---\n\nNotes on Windows:\n\nhttp://blogs.msdn.com/b/oldnewthing/archive/2005/03/08/389527.aspx\n- there are typically two 8-bit code pages active, the so-called \"ANSI\" code page and the so-called \"OEM\" code page\n- cmd.exe `/U` forces Unicode (UTF-16LE) for piped/redirected output\n - can use `chcp 65001` to force the console codepage to utf8 for that session.\n- By default, PowerShell Core (`pwsh`) uses UTF-8 without BOM for redirected output.\n",
"body": "## Status\n\nhttps://github.com/neovim/neovim/pull/33012 takes the first step: default to 'noshelltemp'.\n\nThis will help narrow the surface area of potential problems on Windows.\n\n## Next steps\n\nEven if it regresses something, we can continue with [this next step](https://github.com/neovim/neovim/issues/1008#issuecomment-50602022):\n\n> Find the code vim uses to detect encoding using temporary files\n> Refactor to enable usage of byte strings, and always apply it to the data read by filters\n\nThis may just mean \"ensure `readfile()` is called by `do_filter()` where necessary\", or it may mean we need to explicitly detect encoding in `do_filter()` by some other means.\n\nIf 'shelltemp' becomes a no-op, then these events are also no-ops/deprecated:\n\n> The |FilterReadPre|, |FilterReadPost| and |FilterWritePre|, |FilterWritePost| autocommands event are not triggered when 'shelltemp' is off.",
0 commit comments