In C++, tokens can be defined as the smallest building block of C++ programs that the compiler understands. It is the smallest unit of a program into which the compiler divides the code for further processing. In this article, we will learn about different types of tokens in C++.
Types of Tokens in C++
We have 5 types of tokens each of which serves a specific purpose in the syntax and semantics of C++. Below are the main types of tokens in C++:
1. Identifiers
In C++, entities like variables, functions, classes, or structs must be given unique names within the program so that they can be uniquely identified. The unique names given to these entities are known as identifiers.
It is recommended to choose valid and relevant names of identifiers to write readable and maintainable programs. Keywords cannot be used as an identifier because they are reserved words to do specific tasks. In the below example, "first_name' is an identifier.
string first_name = "Raju";
We have to follow a set of rules to define the name of identifiers as follows:
- An identifier can only begin with a letter or an underscore(_).
- An identifier can consist of letters (A-Z or a-z), digits (0-9), and underscores (_). White spaces and Special characters can not be used as the name of an identifier.
- Keywords cannot be used as an identifier because they are reserved words to do specific tasks. For example, string, int, class, struct, etc.
- Identifier must be unique in its namespace.
- As C++ is a case-sensitive language so identifiers such as 'first_name' and 'First_name' are different entities.
2. Keywords
Keywords in C++ are the tokens that are the reserved words in programming languages that have their specific meaning and functionalities within a program. Keywords cannot be used as an identifier to name any variables.
For example, a variable or function cannot be named as 'int' because it is reserved for declaring integer data type.
There are 95 keywords reserved in C++.
3. Constants
Constants are the tokens in C++ that are used to define variables at the time of initialization and the assigned value cannot be changed after that.We can define the constants in C++ in two ways that are using the const, constexpr keyword and #define preprocessor directive.
const type name = val
constexpr type name = val
#define name val
4. Strings
In C++, a string is not a built-in data type like 'int', 'char', or 'float'. It is a class available in the STL library which provides the functionality to work with a sequence of characters, that represents a string of text.
string variable_name;
string name= "This is string";
When we define any variable using the 'string' keyword we are actually defining an object that represents a sequence of characters. We can perform various methods on the string provided by the string class such as length(), push_backk(), and pop_back().
5. Punctuators
Punctuators are the token characters having specific meanings within the syntax of the programming language. These symbols are used in a variety of functions, including ending the statements, defining control statements, separating items, and more.
Below are the most common punctuators used in C++ programming:
- Semicolon (;): It is used to terminate the statement.
- Square brackets []: They are used to store array elements.
- Curly Braces {}: They are used to define blocks of code.
- Double-quote ("): It is used to enclose string literals.
- Single-quote ('): It is used to enclose character literals.
6. Operators
C++ operators are special symbols that are used to perform operations on operands such as variables, constants, or expressions. A wide range of operators is available in C++ to perform a specific type of operations which includes arithmetic operations, comparison operations, logical operations, and more.
For example, (A+B), in which 'A' and 'B' are operands, and '+' is an arithmetic operator which is used to add two operands.
There can be three types of operators:
1. Unary Operators
Unary operators are used with single operands only. They perform the operations on a single variable. For example, increment and decrement operators.
- Increment operator ( ++ ): It is used to increment the value of an operand by 1.
- Decrement operator ( -- ): It is used to decrement the value of an operand by 1.
2. Binary Operators
They are used with the two operands and they perform the operations between the two variables. For example (A<B), less than (<) operator compares A and B, returns true if A is less than B else returns false.
- Arithmetic Operators: These operators perform basic arithmetic operations on operands. They include '+', '-', '*', '/', and '%'
- Comparison Operators: These operators are used to compare two operands, and they include '==', '!=', '<', '>', '<=', and '>='.
- Logical Operators: These operators perform logical operations on boolean values. They include '&&', '||', and '!'.
- Assignment Operators: These operators are used to assign values to variables, and they include 'variables', '-=', '*=', '/=', and '%='.
- Bitwise Operators: These operators perform bitwise operations on integers. They include'&', '|', '^', '~', '<<', and '>>'.
3. Ternary Operator
The ternary operator is the only operator that takes three operands. It is also known as a conditional operator that is used for conditional expressions.
Expression1 ? Expression2 : Expression3;
IfExpression1 became true then Expression2 will be executed otherwise Expression3 will be executed.
Similar Reads
Literals in C
In C, Literals are the constant values that are assigned to the variables. Literals represent fixed values that cannot be modified. Literals contain memory but they do not have references as variables. Generally, both terms, constants, and literals are used interchangeably. For example, “const int =
4 min read
Literals in C++
In C++ programming language, literals are fundamental elements used to represent fixed values. These values can include numbers, characters, strings, and more. They are generally present as the right operand in the assignment operation.Let's take a look at an example:C++#include <iostream> usi
6 min read
C Identifiers
In C programming, identifiers are the names used to identify variables, functions, arrays, structures, or any other user-defined items. It is a name that uniquely identifies a program element and can be used to refer to it later in the program.Example:C// Creating a variable int val = 10; // Creatin
3 min read
C String Functions
C language provides various built-in functions that can be used for various operations and manipulations on strings. These string functions make it easier to perform tasks such as string copy, concatenation, comparison, length, etc. The <string.h> header file contains these string functions.Th
6 min read
Tokenizing a string in C++
Tokenizing a string denotes splitting a string with respect to some delimiter(s). There are many ways to tokenize a string. In this article four of them are explained:Using stringstreamA stringstream associates a string object with a stream allowing you to read from the string as if it were a stream
4 min read
String Tokenization in C
In C, tokenization is the process of breaking the string into smaller parts using delimiters (characters treated as separators) like space, commas, a specific character, or even a string. Those smaller parts are called tokens where each token is a substring of the original string separated by the de
3 min read
strol() function in C++
The strtol() function in C++ interprets the contents of a string as an integral number of the specified base and return its value as a long int.This function also sets an end pointer that points to the first character after the last valid numeric character of the string, if there is no such characte
3 min read
goto Statement in C
The goto statement in C allows the program to jump to some part of the code, giving you more control over its execution. While it can be useful in certain situations, like error handling or exiting complex loops, it's generally not recommended because it can make the code harder to read and maintain
4 min read
C++ Variables
In C++, variable is a name given to a memory location. It is the basic unit of storage in a program. The value stored in a variable can be accessed or changed during program execution.Creating a VariableCreating a variable and giving it a name is called variable definition (sometimes called variable
4 min read
strcat() in C
C strcat() function appends the string pointed to by src to the end of the string pointed to by dest. It will append a copy of the source string in the destination string. plus a terminating Null character. The initial character of the string(src) overwrites the Null-character present at the end of
2 min read