What tokens are in the end of line?What are the exact semantics of detokenize?Test whether end will really end the runWhy are some characters not allowed in command sequencesHow TeX reads what you typeWhy does the TeX scanner process tokens for register numbers and macro names differently?How to pass a control-sequence as an argument to a macro that expects whitespace after the argument?What state changes are caused by input?A line starting with a spaceHow is the TeX engine informed of the OS's end-of-line marker and file encoding?About what TeX does after it finds commented conditionals with ltxdoc

Find the Factorial From the Given Prime Relationship

Is open-sourcing the code of a webapp not recommended?

Show that this function is bounded

Should an arbiter claim draw at a K+R vs K+R endgame?

Which comes first? Multiple Imputation, Splitting into train/test, or Standardization/Normalization

Preventing Employees from either switching to Competitors or Opening Their Own Business

When 2-pentene reacts with HBr, what will be the major product?

What can plausibly explain many of my very long and low-tech bridges?

Why doesn’t a normal window produce an apparent rainbow?

Is an early checkout possible at a hotel before its reception opens?

Smooth switching between 12 V batteries, with a toggle switch

At what point in time did Dumbledore ask Snape for this favor?

Winning Strategy for the Magician and his Apprentice

What could have caused a rear derailleur to end up in the back wheel suddenly?

Can someone help me to understand this sentence? "Apple Inc. tumbled as the companies appear set to undergo U.S. antitrust probes"

Movie about a boy who was born old and grew young

Payment instructions allegedly from HomeAway look fishy to me

PhD - Well known professor or well known school?

What's the largest optical telescope mirror ever put in space?

Compiling c files on ubuntu and using the executable on Windows

When conversion from Integer to Single may lose precision

How to tell your grandparent to not come to fetch you with their car?

How is water heavier than petrol, even though its molecular weight is less than petrol?

What should the arbiter and what should have I done in this case?



What tokens are in the end of line?


What are the exact semantics of detokenize?Test whether end will really end the runWhy are some characters not allowed in command sequencesHow TeX reads what you typeWhy does the TeX scanner process tokens for register numbers and macro names differently?How to pass a control-sequence as an argument to a macro that expects whitespace after the argument?What state changes are caused by input?A line starting with a spaceHow is the TeX engine informed of the OS's end-of-line marker and file encoding?About what TeX does after it finds commented conditionals with ltxdoc













8















Consider the following test.tex:



ERROR
a


If we run tex on it, and start skipping tokens one by one, we get this:



(test.tex
! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)


As we see, the first 1 command skips end-of-line.
Then we see that TeX has just read a and is about to read end-of-line character. We skip it with the second 1 command.



Then we see the same picture. As if nothing happened. And only after the third 1 command we see that TeX has passed further.



Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?



Also, it is strange that if after the final *, which is not shown in the output, we say end, tex says:



*end
<*> end

?


instead of terminating.



EDIT



Consider the following 6 test files:



control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex



Here are results that I obtained:



control-word_at_eol.tex: space and ^^M is skipped right after reading control word



control-word_not_at_eol.tex: space is not skipped right after reading control word



non-space_at_eol.tex: space is skipped right after reading control symbol, ^^M not skipped



non-space_not_at_eol.tex: space is not skipped right after reading control symbol



space_at_eol.tex: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol



space_not_at_eol.tex: space is not skipped right after reading control symbol










share|improve this question
























  • correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

    – touhami
    May 22 at 23:07















8















Consider the following test.tex:



ERROR
a


If we run tex on it, and start skipping tokens one by one, we get this:



(test.tex
! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)


As we see, the first 1 command skips end-of-line.
Then we see that TeX has just read a and is about to read end-of-line character. We skip it with the second 1 command.



Then we see the same picture. As if nothing happened. And only after the third 1 command we see that TeX has passed further.



Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?



Also, it is strange that if after the final *, which is not shown in the output, we say end, tex says:



*end
<*> end

?


instead of terminating.



EDIT



Consider the following 6 test files:



control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex



Here are results that I obtained:



control-word_at_eol.tex: space and ^^M is skipped right after reading control word



control-word_not_at_eol.tex: space is not skipped right after reading control word



non-space_at_eol.tex: space is skipped right after reading control symbol, ^^M not skipped



non-space_not_at_eol.tex: space is not skipped right after reading control symbol



space_at_eol.tex: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol



space_not_at_eol.tex: space is not skipped right after reading control symbol










share|improve this question
























  • correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

    – touhami
    May 22 at 23:07













8












8








8


0






Consider the following test.tex:



ERROR
a


If we run tex on it, and start skipping tokens one by one, we get this:



(test.tex
! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)


As we see, the first 1 command skips end-of-line.
Then we see that TeX has just read a and is about to read end-of-line character. We skip it with the second 1 command.



Then we see the same picture. As if nothing happened. And only after the third 1 command we see that TeX has passed further.



Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?



Also, it is strange that if after the final *, which is not shown in the output, we say end, tex says:



*end
<*> end

?


instead of terminating.



EDIT



Consider the following 6 test files:



control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex



Here are results that I obtained:



control-word_at_eol.tex: space and ^^M is skipped right after reading control word



control-word_not_at_eol.tex: space is not skipped right after reading control word



non-space_at_eol.tex: space is skipped right after reading control symbol, ^^M not skipped



non-space_not_at_eol.tex: space is not skipped right after reading control symbol



space_at_eol.tex: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol



space_not_at_eol.tex: space is not skipped right after reading control symbol










share|improve this question
















Consider the following test.tex:



ERROR
a


If we run tex on it, and start skipping tokens one by one, we get this:



(test.tex
! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)


As we see, the first 1 command skips end-of-line.
Then we see that TeX has just read a and is about to read end-of-line character. We skip it with the second 1 command.



Then we see the same picture. As if nothing happened. And only after the third 1 command we see that TeX has passed further.



Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?



Also, it is strange that if after the final *, which is not shown in the output, we say end, tex says:



*end
<*> end

?


instead of terminating.



EDIT



Consider the following 6 test files:



control-word_at_eol.tex
control-word_not_at_eol.tex
non-space_at_eol.tex
non-space_not_at_eol.tex
space_at_eol.tex
space_not_at_eol.tex



Here are results that I obtained:



control-word_at_eol.tex: space and ^^M is skipped right after reading control word



control-word_not_at_eol.tex: space is not skipped right after reading control word



non-space_at_eol.tex: space is skipped right after reading control symbol, ^^M not skipped



non-space_not_at_eol.tex: space is not skipped right after reading control symbol



space_at_eol.tex: space, ^^M and (!!!) name of control symbol are skipped right after reading control symbol



space_not_at_eol.tex: space is not skipped right after reading control symbol







tex-core






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited May 22 at 3:32







Igor Liferenko

















asked May 21 at 9:15









Igor LiferenkoIgor Liferenko

2,309829




2,309829












  • correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

    – touhami
    May 22 at 23:07

















  • correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

    – touhami
    May 22 at 23:07
















correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

– touhami
May 22 at 23:07





correction) in file 2) space is skipped, it's ^^M which is not, in file 3) space not skipped it's ^^M. file 6) is same as 2 space skkiped but ^^M not.

– touhami
May 22 at 23:07










2 Answers
2






active

oldest

votes


















11














You can understand this better with a different input



ERROR xyz
a


If you run tex on it, you get



This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz


Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.



With 1 you delete the next token and TeX stops again showing the context as before.



What happens with your attempt?



ERROR
a


Let's see it step-by-step:



! Undefined control sequence.
l.1 ERROR

?


Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M and the generated space has been gobbled during tokenization of ERROR. TeX is in state N (beginning of a new line).



If you type 1 followed by return, you get



? 1
l.2 a

?


TeX has entered state M because it has read a (and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M is still unread.



Type another 1 to get



? 1
l.2 a

?


The gobbled token is now the space generated by ^^M and TeX enters state N again.



Type another 1:



? 1
)
*


The ) means the file has ended; * means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.



Type end:



<*> end

?


Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*> to denote something directly input in the interactive session and what it has ignored following the previous 1 instruction.



Hitting return now shows the * prompt: TeX is awaiting for input.






share|improve this answer

























  • Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

    – Igor Liferenko
    May 22 at 1:17











  • Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

    – Igor Liferenko
    May 22 at 1:47











  • Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

    – Igor Liferenko
    May 22 at 2:58











  • See EDIT in OP.

    – Igor Liferenko
    May 22 at 3:32











  • @IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

    – egreg
    May 22 at 8:43


















10















Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?




TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).



As ^M has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR and which explains the different behaviour for the first two lines.



if you delete 2 tokens then end works



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

?
)
*end
No pages of output.


using pdftex here.



with three 1 you get



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)
*end
<*> end


because the final 1 gobbles the next token but that is the end that you are about to add....






share|improve this answer

























  • The difference is because you do not type 1 after the last ?.

    – Igor Liferenko
    May 21 at 9:38











  • @IgorLiferenko yes I just updated the answer

    – David Carlisle
    May 21 at 9:39











Your Answer








StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "85"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f491891%2fwhat-tokens-are-in-the-end-of-line%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























2 Answers
2






active

oldest

votes








2 Answers
2






active

oldest

votes









active

oldest

votes






active

oldest

votes









11














You can understand this better with a different input



ERROR xyz
a


If you run tex on it, you get



This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz


Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.



With 1 you delete the next token and TeX stops again showing the context as before.



What happens with your attempt?



ERROR
a


Let's see it step-by-step:



! Undefined control sequence.
l.1 ERROR

?


Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M and the generated space has been gobbled during tokenization of ERROR. TeX is in state N (beginning of a new line).



If you type 1 followed by return, you get



? 1
l.2 a

?


TeX has entered state M because it has read a (and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M is still unread.



Type another 1 to get



? 1
l.2 a

?


The gobbled token is now the space generated by ^^M and TeX enters state N again.



Type another 1:



? 1
)
*


The ) means the file has ended; * means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.



Type end:



<*> end

?


Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*> to denote something directly input in the interactive session and what it has ignored following the previous 1 instruction.



Hitting return now shows the * prompt: TeX is awaiting for input.






share|improve this answer

























  • Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

    – Igor Liferenko
    May 22 at 1:17











  • Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

    – Igor Liferenko
    May 22 at 1:47











  • Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

    – Igor Liferenko
    May 22 at 2:58











  • See EDIT in OP.

    – Igor Liferenko
    May 22 at 3:32











  • @IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

    – egreg
    May 22 at 8:43















11














You can understand this better with a different input



ERROR xyz
a


If you run tex on it, you get



This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz


Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.



With 1 you delete the next token and TeX stops again showing the context as before.



What happens with your attempt?



ERROR
a


Let's see it step-by-step:



! Undefined control sequence.
l.1 ERROR

?


Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M and the generated space has been gobbled during tokenization of ERROR. TeX is in state N (beginning of a new line).



If you type 1 followed by return, you get



? 1
l.2 a

?


TeX has entered state M because it has read a (and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M is still unread.



Type another 1 to get



? 1
l.2 a

?


The gobbled token is now the space generated by ^^M and TeX enters state N again.



Type another 1:



? 1
)
*


The ) means the file has ended; * means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.



Type end:



<*> end

?


Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*> to denote something directly input in the interactive session and what it has ignored following the previous 1 instruction.



Hitting return now shows the * prompt: TeX is awaiting for input.






share|improve this answer

























  • Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

    – Igor Liferenko
    May 22 at 1:17











  • Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

    – Igor Liferenko
    May 22 at 1:47











  • Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

    – Igor Liferenko
    May 22 at 2:58











  • See EDIT in OP.

    – Igor Liferenko
    May 22 at 3:32











  • @IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

    – egreg
    May 22 at 8:43













11












11








11







You can understand this better with a different input



ERROR xyz
a


If you run tex on it, you get



This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz


Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.



With 1 you delete the next token and TeX stops again showing the context as before.



What happens with your attempt?



ERROR
a


Let's see it step-by-step:



! Undefined control sequence.
l.1 ERROR

?


Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M and the generated space has been gobbled during tokenization of ERROR. TeX is in state N (beginning of a new line).



If you type 1 followed by return, you get



? 1
l.2 a

?


TeX has entered state M because it has read a (and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M is still unread.



Type another 1 to get



? 1
l.2 a

?


The gobbled token is now the space generated by ^^M and TeX enters state N again.



Type another 1:



? 1
)
*


The ) means the file has ended; * means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.



Type end:



<*> end

?


Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*> to denote something directly input in the interactive session and what it has ignored following the previous 1 instruction.



Hitting return now shows the * prompt: TeX is awaiting for input.






share|improve this answer















You can understand this better with a different input



ERROR xyz
a


If you run tex on it, you get



This is TeX, Version 3.14159265 (TeX Live 2019) (preloaded format=tex)
(./igor.tex
! Undefined control sequence.
l.1 ERROR
xyz
? 1
l.1 ERROR x
yz


Error lines are divided into two parts (actually three if TeX is in the middle of expansions when it finds an error); here the top line shows how far TeX has gone in the input file, the bottom line shows what's awaiting to be read. Note that ERROR sits in the top line, so it has already been read and gone. The slight misalignment between the two lines is because TeX represents control words with a trailing space, but the space “is not there” and, anyway, it is no longer relevant because it's in the top line.



With 1 you delete the next token and TeX stops again showing the context as before.



What happens with your attempt?



ERROR
a


Let's see it step-by-step:



! Undefined control sequence.
l.1 ERROR

?


Nothing is shown in the bottom line, because the line has ended. The end-of-line has already been converted to a category code ^^M and the generated space has been gobbled during tokenization of ERROR. TeX is in state N (beginning of a new line).



If you type 1 followed by return, you get



? 1
l.2 a

?


TeX has entered state M because it has read a (and ignored it); the bottom line shows nothing for the same reason as before. Note that the ^^M is still unread.



Type another 1 to get



? 1
l.2 a

?


The gobbled token is now the space generated by ^^M and TeX enters state N again.



Type another 1:



? 1
)
*


The ) means the file has ended; * means that TeX is awaiting for input. This is again in the format “top line/bottom line”: there was nothing to be ignored, but TeX follows instructions and wants to ignore something.



Type end:



<*> end

?


Again, “top line/bottom line”: the top line shows what has been read in so far. There is no line number, hence TeX shows <*> to denote something directly input in the interactive session and what it has ignored following the previous 1 instruction.



Hitting return now shows the * prompt: TeX is awaiting for input.







share|improve this answer














share|improve this answer



share|improve this answer








edited May 22 at 8:42

























answered May 21 at 11:01









egregegreg

746k8919523293




746k8919523293












  • Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

    – Igor Liferenko
    May 22 at 1:17











  • Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

    – Igor Liferenko
    May 22 at 1:47











  • Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

    – Igor Liferenko
    May 22 at 2:58











  • See EDIT in OP.

    – Igor Liferenko
    May 22 at 3:32











  • @IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

    – egreg
    May 22 at 8:43

















  • Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

    – Igor Liferenko
    May 22 at 1:17











  • Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

    – Igor Liferenko
    May 22 at 1:47











  • Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

    – Igor Liferenko
    May 22 at 2:58











  • See EDIT in OP.

    – Igor Liferenko
    May 22 at 3:32











  • @IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

    – egreg
    May 22 at 8:43
















Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

– Igor Liferenko
May 22 at 1:17





Top line does not end with a space in "xyz" example: try double click in terminal at the end of top line - highlighting will begin right after ERROR, thus there is no space.

– Igor Liferenko
May 22 at 1:17













Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

– Igor Liferenko
May 22 at 1:47





Ideally, in the original example the ^^M converted to space should have been shown at the end of top line after second 1 command (after a), because the top line shows how far TeX has gone in the input file. The absence of space can be verified objectively by running hexdump -Cv test.log

– Igor Liferenko
May 22 at 1:47













Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

– Igor Liferenko
May 22 at 2:58





Please fix "The top line ends with a space" sentence, as it is wrong. See answer to exercise 8.3 in TeXbook.

– Igor Liferenko
May 22 at 2:58













See EDIT in OP.

– Igor Liferenko
May 22 at 3:32





See EDIT in OP.

– Igor Liferenko
May 22 at 3:32













@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

– egreg
May 22 at 8:43





@IgorLiferenko It was not wrong; I reworded anyway. About the six files, I don't know what to say: I'm not going to download them from an unknown repository

– egreg
May 22 at 8:43











10















Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?




TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).



As ^M has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR and which explains the different behaviour for the first two lines.



if you delete 2 tokens then end works



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

?
)
*end
No pages of output.


using pdftex here.



with three 1 you get



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)
*end
<*> end


because the final 1 gobbles the next token but that is the end that you are about to add....






share|improve this answer

























  • The difference is because you do not type 1 after the last ?.

    – Igor Liferenko
    May 21 at 9:38











  • @IgorLiferenko yes I just updated the answer

    – David Carlisle
    May 21 at 9:39















10















Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?




TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).



As ^M has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR and which explains the different behaviour for the first two lines.



if you delete 2 tokens then end works



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

?
)
*end
No pages of output.


using pdftex here.



with three 1 you get



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)
*end
<*> end


because the final 1 gobbles the next token but that is the end that you are about to add....






share|improve this answer

























  • The difference is because you do not type 1 after the last ?.

    – Igor Liferenko
    May 21 at 9:38











  • @IgorLiferenko yes I just updated the answer

    – David Carlisle
    May 21 at 9:39













10












10








10








Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?




TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).



As ^M has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR and which explains the different behaviour for the first two lines.



if you delete 2 tokens then end works



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

?
)
*end
No pages of output.


using pdftex here.



with three 1 you get



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)
*end
<*> end


because the final 1 gobbles the next token but that is the end that you are about to add....






share|improve this answer
















Why TeX skips end-of-line after ERROR from the first time, but after a only from the second time?




TeX removes any spaces from the end of the line (some older implementations erroneously removed tabs as well) and then if endlinechar is in the 0-255 range that character is added to the end of the line) in plain and latex that defaults to 13 (control M).



As ^M has catcode 5, it normally acts as space so is gobbled while tokenizing ERROR and which explains the different behaviour for the first two lines.



if you delete 2 tokens then end works



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

?
)
*end
No pages of output.


using pdftex here.



with three 1 you get



! Undefined control sequence.
l.1 ERROR

? 1
l.2 a

? 1
l.2 a

? 1
)
*end
<*> end


because the final 1 gobbles the next token but that is the end that you are about to add....







share|improve this answer














share|improve this answer



share|improve this answer








edited May 21 at 9:44

























answered May 21 at 9:35









David CarlisleDavid Carlisle

508k4311551908




508k4311551908












  • The difference is because you do not type 1 after the last ?.

    – Igor Liferenko
    May 21 at 9:38











  • @IgorLiferenko yes I just updated the answer

    – David Carlisle
    May 21 at 9:39

















  • The difference is because you do not type 1 after the last ?.

    – Igor Liferenko
    May 21 at 9:38











  • @IgorLiferenko yes I just updated the answer

    – David Carlisle
    May 21 at 9:39
















The difference is because you do not type 1 after the last ?.

– Igor Liferenko
May 21 at 9:38





The difference is because you do not type 1 after the last ?.

– Igor Liferenko
May 21 at 9:38













@IgorLiferenko yes I just updated the answer

– David Carlisle
May 21 at 9:39





@IgorLiferenko yes I just updated the answer

– David Carlisle
May 21 at 9:39

















draft saved

draft discarded
















































Thanks for contributing an answer to TeX - LaTeX Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f491891%2fwhat-tokens-are-in-the-end-of-line%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Club Baloncesto Breogán Índice Historia | Pavillón | Nome | O Breogán na cultura popular | Xogadores | Adestradores | Presidentes | Palmarés | Historial | Líderes | Notas | Véxase tamén | Menú de navegacióncbbreogan.galCadroGuía oficial da ACB 2009-10, páxina 201Guía oficial ACB 1992, páxina 183. Editorial DB.É de 6.500 espectadores sentados axeitándose á última normativa"Estudiantes Junior, entre as mellores canteiras"o orixinalHemeroteca El Mundo Deportivo, 16 setembro de 1970, páxina 12Historia do BreogánAlfredo Pérez, o último canoneiroHistoria C.B. BreogánHemeroteca de El Mundo DeportivoJimmy Wright, norteamericano do Breogán deixará Lugo por ameazas de morteResultados de Breogán en 1986-87Resultados de Breogán en 1990-91Ficha de Velimir Perasović en acb.comResultados de Breogán en 1994-95Breogán arrasa al Barça. "El Mundo Deportivo", 27 de setembro de 1999, páxina 58CB Breogán - FC BarcelonaA FEB invita a participar nunha nova Liga EuropeaCharlie Bell na prensa estatalMáximos anotadores 2005Tempada 2005-06 : Tódolos Xogadores da Xornada""Non quero pensar nunha man negra, mais pregúntome que está a pasar""o orixinalRaúl López, orgulloso dos xogadores, presume da boa saúde económica do BreogánJulio González confirma que cesa como presidente del BreogánHomenaxe a Lisardo GómezA tempada do rexurdimento celesteEntrevista a Lisardo GómezEl COB dinamita el Pazo para forzar el quinto (69-73)Cafés Candelas, patrocinador del CB Breogán"Suso Lázare, novo presidente do Breogán"o orixinalCafés Candelas Breogán firma el mayor triunfo de la historiaEl Breogán realizará 17 homenajes por su cincuenta aniversario"O Breogán honra ao seu fundador e primeiro presidente"o orixinalMiguel Giao recibiu a homenaxe do PazoHomenaxe aos primeiros gladiadores celestesO home que nos amosa como ver o Breo co corazónTita Franco será homenaxeada polos #50anosdeBreoJulio Vila recibirá unha homenaxe in memoriam polos #50anosdeBreo"O Breogán homenaxeará aos seus aboados máis veteráns"Pechada ovación a «Capi» Sanmartín e Ricardo «Corazón de González»Homenaxe por décadas de informaciónPaco García volve ao Pazo con motivo do 50 aniversario"Resultados y clasificaciones""O Cafés Candelas Breogán, campión da Copa Princesa""O Cafés Candelas Breogán, equipo ACB"C.B. Breogán"Proxecto social"o orixinal"Centros asociados"o orixinalFicha en imdb.comMario Camus trata la recuperación del amor en 'La vieja música', su última película"Páxina web oficial""Club Baloncesto Breogán""C. B. Breogán S.A.D."eehttp://www.fegaba.com

Vilaño, A Laracha Índice Patrimonio | Lugares e parroquias | Véxase tamén | Menú de navegación43°14′52″N 8°36′03″O / 43.24775, -8.60070

Cegueira Índice Epidemioloxía | Deficiencia visual | Tipos de cegueira | Principais causas de cegueira | Tratamento | Técnicas de adaptación e axudas | Vida dos cegos | Primeiros auxilios | Crenzas respecto das persoas cegas | Crenzas das persoas cegas | O neno deficiente visual | Aspectos psicolóxicos da cegueira | Notas | Véxase tamén | Menú de navegación54.054.154.436928256blindnessDicionario da Real Academia GalegaPortal das Palabras"International Standards: Visual Standards — Aspects and Ranges of Vision Loss with Emphasis on Population Surveys.""Visual impairment and blindness""Presentan un plan para previr a cegueira"o orixinalACCDV Associació Catalana de Cecs i Disminuïts Visuals - PMFTrachoma"Effect of gene therapy on visual function in Leber's congenital amaurosis"1844137110.1056/NEJMoa0802268Cans guía - os mellores amigos dos cegosArquivadoEscola de cans guía para cegos en Mortágua, PortugalArquivado"Tecnología para ciegos y deficientes visuales. Recopilación de recursos gratuitos en la Red""Colorino""‘COL.diesis’, escuchar los sonidos del color""COL.diesis: Transforming Colour into Melody and Implementing the Result in a Colour Sensor Device"o orixinal"Sistema de desarrollo de sinestesia color-sonido para invidentes utilizando un protocolo de audio""Enseñanza táctil - geometría y color. Juegos didácticos para niños ciegos y videntes""Sistema Constanz"L'ocupació laboral dels cecs a l'Estat espanyol està pràcticament equiparada a la de les persones amb visió, entrevista amb Pedro ZuritaONCE (Organización Nacional de Cegos de España)Prevención da cegueiraDescrición de deficiencias visuais (Disc@pnet)Braillín, un boneco atractivo para calquera neno, con ou sen discapacidade, que permite familiarizarse co sistema de escritura e lectura brailleAxudas Técnicas36838ID00897494007150-90057129528256DOID:1432HP:0000618D001766C10.597.751.941.162C97109C0155020