mirrors/eiffel-org
1
0
Fork 0
mirror of https://github.com/EiffelSoftware/eiffel-org.git synced 2026-02-11 04:55:14 +01:00
Code Issues Packages Projects Releases Wiki Activity

Author:admin

Browse Source 
Date:2008-09-19T07:54:43.000000Z


git-svn-id: https://svn.eiffel.com/eiffel-org/trunk@25 abb3cda0-5349-4a8f-a601-0c33ac3a8c38
This commit is contained in:
jfiat
2008-09-19 07:54:43 +00:00
parent c0aba35d72
commit 8a46c5d793
247 changed files with 1257 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
documentation/current/solutions/text-processing/eiffellex/eiffellex-tutorial.wiki
View File

@@ -1,6 +1,7 @@
[[Property:title|Lexical analysis: the Lex library]]
[[Property:link_title|EiffelLex Tutorial]]
[[Property:weight|0]]
[[Property:uuid|9ea43bef-1483-fbf2-4791-2be6a31d394d]]
==OVERVIEW==
When analyzing a text by computer, it is usually necessary to split it into individual components or '''tokens'''. In human languages, the tokens are the words; in programming languages, tokens are the basic constituents of software texts, such as identifiers, constants and special symbols.