%% This is part of the OpTeX project, see http://petr.olsak.net/optex \_codedecl \begtt {Verbatim <2022-04-23>} % preloaded in format \_doc ---------------------------- The internal parameters \`\_ttskip`, \`\_ttpenalty`, \`\_viline`, \`\_vifile` and \`\_ttfont` for verbatim macros are set. \_cod ---------------------------- \_def\_ttskip{\_medskip} % space above and below \begtt, \verbinput \_mathchardef\_ttpenalty=100 % penalty between lines in \begtt, \verbinput \_newcount\_viline % last line number in \verbinput \_newread\_vifile % file given by \verinput \_def\_ttfont{\_tt} % default tt font \_doc ---------------------------- \`\code``{}` expands to `\detokenize{}` when `\escapechar=-1`. In order to do it more robust when it is used in `\write` then it expands as noexpanded `\code` (followed by space in its csname). This macro does the real work. The \`\_printinverbatim``{}` macro is used for `\code{}` printing and for \code{`}\code{`} printing. It is defined as `\hbox`, so the in-verbatim will be never broken. But you can re-define this macro. When `\code` occurs in PDF outlines then it does the same as `\detokenize`. The macro for preparing outlines sets `\escapechar` to $-1$ and uses \^`\_regoul` token list before `\edef`. The `\code` is not `\proteced` because we want it expands to `\unexpanded{\code{}}` in `\write` parameters. This protect the expansions of the `\code` parameter (like `\\`, `\^` etc.). \_cod ---------------------------- \_def\_code#1{\_unexpanded\_ea{\_csname _code \_endcsname{#1}}} \_protected\_sdef{_code }#1{{\_escapechar=-1 \_ttfont \_the\_everyintt \_relax \_ea\_printinverbatim\_ea{\_detokenize{#1}}}} \_def\_printinverbatim#1{\_leavevmode\_hbox{#1}} \_regmacro {}{}{\_let\code=\_detokenize \_let\_code=\_detokenize} \_public \code ; \_doc ---------------------------- The \`\_setverb` macro sets all catcodes to \"verbatim mode". It should be used only in a group, so we prepare a new catcode table with \"verbatim" catcodes and we define it as\nl `\_catcodetable`\`\_verbatimcatcodes`. After the group is finished then original catcode table is restored. \_cod ---------------------------- \_newcatcodetable \_verbatimcatcodes \_def\_setverb{\_begingroup \_def\do##1{\_catcode`##1=12 } \_dospecials \_savecatcodetable\_verbatimcatcodes % all characters are normal \_endgroup } \_setverb \_def\_setverb{\_catcodetable\_verbatimcatcodes }% \_doc ---------------------------- \`\verbchar``` saves original catcode of previously declared `` (if such character was declared) using \`\_savedttchar` and \`\_savedttcharc` values. Then new such values are stored. The declared character is activated by `\_adef` as a macro (active character) which opens a group, does `\_setverb` and other settings and reads its parameter until second the same character. This is done by the \`\_readverb` macro. Finally, it prints scanned `` by \^`\_printinverbatim` and closes group. Suppose that `\verbchar"` is used. Then the following work is schematically done: \begtt \_def "{\_begingroup \_setverb ... \_readverb} \_def \_readverb #1"{\_printinverbatim{#1}\_endgroup} \endtt Note that the second occurrence of `"` is not active because `\_setverb` deactivates it. \_cod ---------------------------- \_def\_verbchar#1{% \_ifx\_savedttchar\_undefined\_else \_catcode\_savedttchar=\_savedttcharc \_fi \_chardef\_savedttchar=`#1 \_chardef\_savedttcharc=\_catcode`#1 \_adef{#1}{\_begingroup \_setverb \_adef{ }{\_dsp}\_ttfont \_the\_everyintt\_relax \_readverb}% \_def\_readverb ##1#1{\_printinverbatim{##1}\_endgroup}% } \_let \_activettchar=\_verbchar % for backward compatibility \_public \verbchar \activettchar ; \_doc ---------------------------- \`\begtt` is defined only as public. We don't need a private `\_begtt` variant. This macro opens a group and sets `%` as an active character (temporary). This will allow it to be used as the comment character at the same line after `\begtt`. Then \`\_begtti` is run. It is defined by \^`\eoldef`, so users can put a parameter at the same line where `\begtt` is. This `#1` parameter is used after \^`\everytt` parameters settings, so users can change them locally. The \^`\_begtti` macro does \^`\_setverb` and another preprocessing, sets `\endlinechar` to `^^J` and reads the following text in verbatim mode until \`\endtt` occurs. This scanning is done by \`\_startverb` macro which is defined as: \begtt \adef/{\bslash} \_def\_startverb #1/endtt #2^^J{...} \endtt We must to ensure that the backslash in `\endtt` has category 12 (this is a reason of the `\ea` chain in real code). The `#2` is something between `\endtt` and the end of the same line and it is simply ignored. The `\_startverb` puts the scanned data to \`\_prepareverbdata`. It sets the data to `\_tmpb` without changes by default, but you should re-define it in order to do special changes if you want. (For example, \^`\hisyntax` redefines this macro.) The scanned data have `^^J` at each end of line and all spaces are active characters (defined as {\visiblesp`\ `}). Other characters have normal category 11 or 12. The `^^J` is appended to verbatim data because we need to be sure that the data are finished by this character. When `\endtt` is preceded by spaces then we need to close these spaces by `^^J` and such line is not printed due to a trick used in \^`\_printverb`. When `\_prepareverbdata` finishes then `\_startverb` runs `\_printverb` loop over each line of the data and does a final work: last skip plus `\noindent` in the next paragraph. \_cod --------------------------- \_def\begtt{\_par \_begingroup \_adef\%##1\_relax{\_relax}\_begtti} \_eoldef \_begtti#1{\_wipeepar \_setxhsize \_vskip\_parskip \_ttskip \_setverb \_ifnum\_ttline<0 \_let\_printverblinenum=\_relax \_else \_initverblinenum \_fi \_adef{ }{\_dsp}\_adef\^^I{\t}\_parindent=\_ttindent \_parskip=0pt \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}% \_protrudechars=0 % disable protrusion \_the\_everytt \_relax #1\_relax \_ttfont \_def\_testcommentchars##1\_iftrue{\_iffalse}\_let\_hicomments=\_relax \_savemathsb \_endlinechar=`^^J \_startverb } \_ea\_def\_ea\_startverb \_ea#\_ea1\_csstring\\endtt#2^^J{% \_prepareverbdata\_tmpb{#1^^J}% \_ea\_printverb \_tmpb\_fin \_par \_restoremathsb \_endgroup \_ttskip \_isnextchar\_par{}{\_noindent}% } \_def\_prepareverbdata#1#2{\_def#1{#2}} \_doc The \`\_printverb` macro calls \^`\_printverbline``{}` repeatedly to each scanned line of verbatim text. The `\_printverb` is used from `\begtt...\endtt` and from `\verbinput` too. The \^`\_testcommentchars` replaces the following `\_iftrue` to `\_iffalse` by default unless the \~`\commentchars` are set. So, the main body of the loop is written in the `\_else` part of the `\_iftrue` condition. The \^`\_printverbline``{}` is called here. The \`\_printverbline``{}` expects that it starts in vertical mode and it must do `\par` to return the vertical mode. The \`\_printverblinenum` is used here: it does nothing when `\_ttline`\code{<0} else it prints the line number using `\_llap`. \`\_putttpenalty` puts \^`\_ttpenalty` before second and next lines, but not before first line in each `\begtt...\endtt` environment. The `\_ttline` is increased here in the `\_printverb` macro because of comments-blocks: the `\_prinverbline` is not processed in comments-blocks but we need to count the `\_ttline`. \_cod ---------------------------- \_def\_printverb #1^^J#2{% \_ifx\_printverblinenum\_relax \_else \_incr\_ttline \_fi \_testcommentchars #1\_relax\_relax\_relax \_iftrue \_ifx\_fin#2\_printcomments\_fi \_else \_ifx\_vcomments\_empty\_else \_printcomments \_def\_vcomments{}\_fi \_ifx\_fin#2% \_bgroup \_adef{ }{}\_def\t{}% if the last line is emtpy, we don't print it \_ifcat&\_egroup \_ifx\_printverblinenum\_relax \_else \_decr\_ttline \_fi \_else\_egroup \_printverbline{#1}\_fi \_else \_printverbline{#1}% \_fi \_fi \_unless\_ifx\_fin#2\_afterfi{\_printverb#2}\_fi } \_def\_printverbline#1{\_putttpenalty \_indent \_printverblinenum \_kern\_ttshift #1\_par} \_def\_initverblinenum{\_tenrm \_thefontscale[700]\_ea\_let\_ea\_sevenrm\_the\_font} \_def\_printverblinenum{\_llap{\_sevenrm \_the\_ttline\_kern.9em}} \_def\_putttpenalty{\_def\_putttpenalty{\_penalty\_ttpenalty}} \_doc ---------------------------- Macro \`\verbinput` uses a file read previously or opens the given file. Then it runs the parameter scanning by \`\_viscanparameter` and \`\_viscanminus`. Finally the \`\_doverbinput` is run. At the beginning of `\_doverbinput`, we have `\_viline`= number of lines already read using previous `\verbinput`, `\_vinolines`= the number of lines we need to skip and `\_vidolnes`= the number of lines we need to print. A similar preparation is done as in `\begtt` after the group is opened. Then we skip \`\_vinolines` lines in a loop a and we read \`\_vidolines` lines. The read data is accumulated into `\_tmpb` macro. The next steps are equal to the steps done in \^`\_startverb` macro: data are processed via \^`\_prepareverbdata` and printed via \^`\_printverb` loop. \_cod \_fin ---------------------- \_def\_verbinput #1(#2) #3 {\_par \_def\_tmpa{#3}% \_def\_tmpb{#1}% cmds used in local group \_ifx\_vifilename\_tmpa \_else \_openin\_vifile={#3}% \_global\_viline=0 \_global\_let\_vifilename=\_tmpa \_ifeof\_vifile \_opwarning{\_string\verbinput: file "#3" unable to read} \_ea\_ea\_ea\_skiptorelax \_fi \_fi \_viscanparameter #2+\_relax } \_def\_skiptorelax#1\_relax{} \_def \_viscanparameter #1+#2\_relax{% \_if$#2$\_viscanminus(#1)\_else \_viscanplus(#1+#2)\_fi } \_def\_viscanplus(#1+#2+){% \_if$#1$\_tmpnum=\_viline \_else \_ifnum#1<0 \_tmpnum=\_viline \_advance\_tmpnum by-#1 \_else \_tmpnum=#1 \_advance\_tmpnum by-1 \_ifnum\_tmpnum<0 \_tmpnum=0 \_fi % (0+13) = (1+13) \_fi \_fi \_edef\_vinolines{\_the\_tmpnum}% \_if$#2$\_def\_vidolines{0}\_else\_edef\_vidolines{#2}\_fi \_doverbinput } \_def\_viscanminus(#1-#2){% \_if$#1$\_tmpnum=0 \_else \_tmpnum=#1 \_advance\_tmpnum by-1 \_fi \_ifnum\_tmpnum<0 \_tmpnum=0 \_fi % (0-13) = (1-13) \_edef\_vinolines{\_the\_tmpnum}% \_if$#2$\_tmpnum=0 \_else \_tmpnum=#2 \_advance\_tmpnum by-\_vinolines \_fi \_edef\_vidolines{\_the\_tmpnum}% \_doverbinput } \_def\_doverbinput{% \_tmpnum=\_vinolines \_advance\_tmpnum by-\_viline \_ifnum\_tmpnum<0 \_openin\_vifile={\_vifilename}% \_global\_viline=0 \_else \_edef\_vinolines{\_the\_tmpnum}% \_fi \_vskip\_parskip \_ttskip \_wipeepar \_setxhsize \_begingroup \_ifnum\_ttline<-1 \_let\_printverblinenum=\_relax \_else \_initverblinenum \_fi \_setverb \_adef{ }{\_dsp}\_adef\^^I{\t}\_parindent=\_ttindent \_parskip=0pt \_def\t{\_hskip \_dimexpr\_tabspaces em/2\_relax}% \_protrudechars=0 % disable protrusion \_the\_everytt\_relax \_tmpb\_relax \_ttfont \_savemathsb \_endlinechar=`^^J \_tmpnum=0 \_loop \_ifeof\_vifile \_tmpnum=\_vinolines\_space \_fi \_ifnum\_tmpnum<\_vinolines\_space \_vireadline \_advance\_tmpnum by1 \_repeat %% skip lines \_edef\_ttlinesave{\_global\_ttline=\_the\_ttline}% \_ifnum\_ttline=-1 \_ttline=\_viline \_else \_let\_ttlinesave=\_relax \_fi \_tmpnum=0 \_def\_tmpb{}% \_ifnum\_vidolines=0 \_tmpnum=-1 \_fi \_ifeof\_vifile \_tmpnum=\_vidolines\_space \_fi \_loop \_ifnum\_tmpnum<\_vidolines\_space \_vireadline \_ifnum\_vidolines=0 \_else\_advance\_tmpnum by1 \_fi \_ifeof\_vifile \_tmpnum=\_vidolines\_space \_else \_visaveline \_fi %% save line \_repeat \_ea\_prepareverbdata \_ea \_tmpb\_ea{\_tmpb^^J}% \_catcode`\ =10 \_catcode`\%=9 % used in \commentchars comments \_ea\_printverb \_tmpb\_fin \_ttlinesave \_par \_restoremathsb \_endgroup \_ttskip \_isnextchar\_par{}{\_noindent}% } \_def\_vireadline{\_read\_vifile to \_tmp \_incr\_viline } \_def\_visaveline{\_ea\_addto\_ea\_tmpb\_ea{\_tmp}} \_public \verbinput ; \_doc ----------------------------- \`\_savemathsb`, \`\_restoremathsb` pair is used in \^`\begtt`...\^`\endtt` or in \^`\verbinput` to temporary suppress the \^`\mathsbon` because we don't need to print `\int _a` in verbatim mode if `\int``_a` is really written. The \^`\_restoremathsb` is defined locally as \^`\mathsbon` only if it is needed. \_cod ----------------------------- \_def\_savemathsb{\_ifmathsb \_mathsboff \_def\_restoremathsb{\_mathsbon}\_fi} \_def\_restoremathsb{} \_doc ----------------------------- If the language of your code\label[commentchars]\wlabel{} printed by \^`\verbinput` supports the format of comments started by two characters from the beginning of the line then you can set these characters by \^`\commentchars```. Such comments are printed in the non-verbatim mode without these two characters and they look like the verbatim printing is interrupted at the places where such comments are. See the section~\ref[lua] for good illustration. The file `optex.lua` is read by a single command `\verbinput (4-) optex.lua` here and the `\commentchars --` was set before it. If you need to set a special character by \^`\commentchars` then you must to set the catcode to 12 (and space to 13). Examples: \begtt \commentchars // % C++ comments \commentchars -- % Lua comments {\catcode`\%=12 \_ea}\commentchars %% % TeX comments {\catcode`\#=12 \catcode`\ =13 \_ea}\commentchars#{ } % bash comments \endtt There is one limitation when \TeX/ interprets the comments declared by \^`\commentchars`. Each block of comments is accumulated to one line and then it is re-interpreted by \TeX. So, the ends of lines in the comments block are lost. You cannot use macros which need to scan end of lines, for example `\begtt...\endtt` inside the comments. The character `%` is ignored in comments but you can use `\%` for printing or `%` alone for de-activating `\_endpar` from empty comment lines. Implementation: The \`\commentchars``` redefines the \`\_testcommentchars` used in \^`\_printverb` in order to it removes the following `\_iftrue` and returns `\_iftrue` or `\_iffalse` depending on the fact that the comment characters are or aren't present at the beginning of tested line. If it is true (`\ifnum` expands to `\ifnum 10>0`) then the rest of the line is added to the \`\_vcomments` macro. The \`\_hicomments` is `\relax` by default but it is redefined by \^`\commentchars` in order to keep no-colorized comments if we need to use feature from \^`\commentchars`. The accumulated comments are printed whenever the non-comment line occurs. This is done by \`\_printcomments` macro. You can re-define it, but the main idea must be kept: it is printed in the group, `\_reloding \_rm` initializes normal font, `\catcodetable0` returns to normal catcode table used before `\verbinput` is started, and the text accumulated in `\_vcomments` must be printed by `\_scantextokens` primitive. \_cod ----------------------------- \_def\_vcomments{} \_let\_hicomments=\_relax \_def\_commentchars#1#2{% \_def\_testcommentchars ##1##2##3\_relax ##4\_iftrue{\_ifnum % not closed in this macro \_ifx #1##1\_ifx#2##21\_fi\_fi 0>0 \_ifx\_relax##3\_relax \_addto\_vcomments{\_endgraf}% empty comment=\enfgraf \_else \_addto\_vcomments{##3 }\_fi}% \_def\_hicomments{\_replfromto{\b\n#1#2}{^^J}{\w{#1#2####1}^^J}}% used in \hisyntax } \_def\_testcommentchars #1\_iftrue{\_iffalse} % default value of \_testcommentchar \_def\_printcomments{\_ttskip {\_catcodetable0 \_rm \_everypar={}% \_noindent \_ignorespaces \_scantextokens\_ea{\_vcomments}\_par}% \_ttskip } \_public \commentchars ; \_doc ----------------------------- The \`\visiblesp` sets spaces as visible characters \char9251. It redefines the \^`\_dsp`, so it is useful for verbatim modes only. The \`\_dsp` is equivalent to {\visiblesp`\ `} primitive. It is used in all verbatim environments: spaces are active and defined as `\_dsp` here. \_cod ----------------------------- \_def \_visiblesp{\_ifx\_initunifonts\_relax \_def\_dsp{\_char9251 }% \_else \_def\_dsp{\_char32 }\_fi} \_let\_dsp=\ % primitive "direct space" \_public \visiblesp ; \_endcode \_endinput History: 2022-04-23 ... \_ttline counting: bug fixed 2022-02-22 ... \_reloading removed due to changes in font-select 2021-04-18 ... \_protrudechars=0 added 2021-04-07 ... \_savemathsb, \_restoremathsb introduced 2021-01-22 ... \activettchar changed to \verbchar 2020-12-30 ... \secc followed by \begtt must be unbreakable 2020-11-13 ... \commentchars implemented 2020-04-22 ... \ttshift introduced 2020-04-06 ... \visiblesp added 2020-04-04 ... ^^I activated as \t for multiline verbatim \verbinput (...) , added.