php.net |  support |  documentation |  report a bug |  advanced search |  search howto |  statistics |  random bug |  login

go to bug id or search bugs for

Bug #35655 Missing end-of-line token
Submitted: 2005-12-13 10:52 UTC Modified: 2005-12-13 22:45 UTC
From: tomas_matousek at hotmail dot com Assigned: iliaa (profile)
Status: Closed Package: Scripting Engine problem
PHP Version: 5CVS-2005-12-13 (snap) OS: *
Private report: No CVE-ID: None
View Developer Edit
Welcome! If you don't have a Git account, you can't do anything here.
If you reported this bug, you can edit this bug over here.
(description)
Block user comment
Status: Assign to:
Package:
Bug Type:
Summary:
From: tomas_matousek at hotmail dot com
New email:
PHP Version: OS:

 

 [2005-12-13 10:52 UTC] tomas_matousek at hotmail dot com 
Description:
------------
PHP lexer omits end-of-line characters following T_END_HEREDOC token. This also causes incorrect output of highlight_string() function.

Reproduce code:
---------------
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>';

display_tokens(token_get_all($code));
highlight_string($code);



function display_tokens($tokens)
{
  foreach ($tokens as $token)
  { 
    if (is_array($token))
    {
      echo token_name($token[0]),"\n";
      echo $token[0]," '",htmlentities(addcslashes($token[1],"\n\r\t")),"'\n";
    }
    else
    {
      echo "    '",htmlentities($token),"'\n";
    }  
  }
}

Expected result:
----------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'
T_WHITESPACE
370 '\r\n  '      // '\r\n' is missing in actual result
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Actual result:
--------------
T_INLINE_HTML
311 '\r\n'
T_OPEN_TAG
367 '<?'
T_WHITESPACE
370 ' \r\n  '
T_VARIABLE
309 '$x'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_START_HEREDOC
371 '<<<EOT\r\n'
T_STRING
307 'some'
T_ENCAPSED_AND_WHITESPACE
314 ' '
T_STRING
307 'string'
T_ENCAPSED_AND_WHITESPACE
314 '    \r\n'
T_END_HEREDOC
372 'EOT'         // missing '\r\n' 
T_WHITESPACE
370 '  '
T_VARIABLE
309 '$y'
T_WHITESPACE
370 ' '
    '='
T_WHITESPACE
370 ' '
T_LNUMBER
305 '2'
    ';'
T_WHITESPACE
370 '\r\n'
T_CLOSE_TAG
369 '?>'


<? 
  $x = <<<EOT
some string    
EOT  $y = 2;
?>

Patches

Pull Requests

History

AllCommentsChangesGit/SVN commitsRelated reports
 [2005-12-13 10:54 UTC] tomas_matousek at hotmail dot com 
Correction:
Expected result of highlighting:
$code = '
<? 
  $x = <<<EOT
some string    
EOT
  $y = 2;
?>'

Actual result:
$code = '
<? 
  $x = <<<EOT
some string    
EOT  $y
 = 2;
?>'
 [2005-12-13 11:31 UTC] tomas_matousek at hotmail dot com 
The same result with latest snapshot.
The problem is probably here (file zend_language_scanner.l, around line 1750):

		if (unput_semicolon) {
			unput(';');
		}

I would suggest to replace unputting with 
  yyless(label_len);
which would preserve both ';' and {NEWLINE}.
 [2005-12-13 13:34 UTC] [email protected] 
Ilia, please check it out.
The proposed patch looks good for me.
 [2005-12-13 22:45 UTC] [email protected] 
This bug has been fixed in CVS.

Snapshots of the sources are packaged every three hours; this change
will be in the next snapshot. You can grab the snapshot at
http://snaps.php.net/.
 
Thank you for the report, and for helping us make PHP better.
 
PHP Copyright © 2001-2025 The PHP Group
All rights reserved. 		Last updated: Sun Jul 20 10:00:03 2025 UTC