In C++, tokens can be defined as the smallest building block of C++ programs that the compiler understands. It is the smallest unit of a program into which the compiler divides the code for further processing. In this article, we will learn about different types of tokens in C++.
Types of Tokens in C++
We have 5 types of tokens each of which serves a specific purpose in the syntax and semantics of C++. Below are the main types of tokens in C++:
1. Identifiers
In C++, entities like variables, functions, classes, or structs must be given unique names within the program so that they can be uniquely identified. The unique names given to these entities are known as identifiers.
It is recommended to choose valid and relevant names of identifiers to write readable and maintainable programs. Keywords cannot be used as an identifier because they are reserved words to do specific tasks. In the below example, “first_name‘ is an identifier.
string first_name = “Raju”;
We have to follow a set of rules to define the name of identifiers as follows:
- An identifier can only begin with a letter or an underscore(_).
- An identifier can consist of letters (A-Z or a-z), digits (0-9), and underscores (_). White spaces and Special characters can not be used as the name of an identifier.
- Keywords cannot be used as an identifier because they are reserved words to do specific tasks. For example, string, int, class, struct, etc.
- Identifier must be unique in its namespace.
- As C++ is a case-sensitive language so identifiers such as ‘first_name’ and ‘First_name’ are different entities.
2. Keywords
Keywords in C++ are the tokens that are the reserved words in programming languages that have their specific meaning and functionalities within a program. Keywords cannot be used as an identifier to name any variables.
For example, a variable or function cannot be named as ‘int’ because it is reserved for declaring integer data type.
There are 95 keywords reserved in C++.
3. Constants
Constants are the tokens in C++ that are used to define variables at the time of initialization and the assigned value cannot be changed after that.We can define the constants in C++ in two ways that are using the const, constexpr keyword and #define preprocessor directive.
const type name = val
constexpr type name = val
#define name val
4. Strings
In C++, a string is not a built-in data type like ‘int’, ‘char’, or ‘float’. It is a class available in the STL library which provides the functionality to work with a sequence of characters, that represents a string of text.
string variable_name;
string name= “This is string”;
When we define any variable using the ‘string’ keyword we are actually defining an object that represents a sequence of characters. We can perform various methods on the string provided by the string class such as length(), push_backk(), and pop_back().
5. Punctuators
Punctuators are the token characters having specific meanings within the syntax of the programming language. These symbols are used in a variety of functions, including ending the statements, defining control statements, separating items, and more.
Below are the most common punctuators used in C++ programming:
- Semicolon (;): It is used to terminate the statement.
- Square brackets []: They are used to store array elements.
- Curly Braces {}: They are used to define blocks of code.
- Double-quote (“): It is used to enclose string literals.
- Single-quote (‘): It is used to enclose character literals.
6. Operators
C++ operators are special symbols that are used to perform operations on operands such as variables, constants, or expressions. A wide range of operators is available in C++ to perform a specific type of operations which includes arithmetic operations, comparison operations, logical operations, and more.
For example, (A+B), in which ‘A’ and ‘B’ are operands, and ‘+’ is an arithmetic operator which is used to add two operands.
There can be three types of operators:
1. Unary Operators
Unary operators are used with single operands only. They perform the operations on a single variable. For example, increment and decrement operators.
- Increment operator ( ++ ): It is used to increment the value of an operand by 1.
- Decrement operator ( — ): It is used to decrement the value of an operand by 1.
2. Binary Operators
They are used with the two operands and they perform the operations between the two variables. For example (A<B), less than (<) operator compares A and B, returns true if A is less than B else returns false.
- Arithmetic Operators: These operators perform basic arithmetic operations on operands. They include ‘+’, ‘-‘, ‘*’, ‘/’, and ‘%’
- Comparison Operators: These operators are used to compare two operands, and they include ‘==’, ‘!=’, ‘<‘, ‘>’, ‘<=’, and ‘>=’.
- Logical Operators: These operators perform logical operations on boolean values. They include ‘&&’, ‘||’, and ‘!‘.
- Assignment Operators: These operators are used to assign values to variables, and they include ‘variables‘, ‘-=‘, ‘*=‘, ‘/=‘, and ‘%=’.
- Bitwise Operators: These operators perform bitwise operations on integers. They include‘&’, ‘|’, ‘^’, ‘~’, ‘<<‘, and ‘>>‘.
3. Ternary Operator
The ternary operator is the only operator that takes three operands. It is also known as a conditional operator that is used for conditional expressions.
Expression1 ? Expression2 : Expression3;
IfExpression1 became true then Expression2 will be executed otherwise Expression3 will be executed.
Similar Reads
C++ Pointers
A pointer is a variable that stores the address of another variable. Pointers can be used with any data type, including basic types (e.g., int, char), arrays, and even user-defined types like classes and structures. Create PointerA pointer can be declared in the same way as any other variable but wi
9 min read
Strings in C++
In C++, strings are sequences of characters that are used to store words and text. They are also used to store data, such as numbers and other types of information in the form of text. Strings are provided by <string> header file in the form of std::string class. Creating a StringCreating a st
6 min read
C++ Data Types
Data types specify the type of data that a variable can store. Whenever a variable is defined in C++, the compiler allocates some memory for that variable based on the data type with which it is declared as every data type requires a different amount of memory. C++ supports a wide variety of data ty
8 min read
Literals In C++
In C++ programming language, literals are fundamental elements used to represent fixed values. These values can include numbers, characters, strings, and more. They are generally present as the right operand in the assignment operation. Let's take a look at an example: [GFGTABS] C++ #include <ios
6 min read
C Identifiers
In C programming, identifiers are the names used to identify variables, functions, arrays, structures, or any other user-defined items. It is a name that uniquely identifies a program element and can be used to refer to it later in the program. Example: [GFGTABS] C // Creating a variable int val = 1
4 min read
C++ Identifiers
In C++ programming language, identifiers are the unique names assigned to variables, functions, classes, structs, or other entities within the program. Let's take a look at an example: [GFGTABS] C++ // Creating a variable int val = 10; // Creating a function void func() {} [/GFGTABS]In the above cod
3 min read
C String Functions
C language provides various built-in functions that can be used for various operations and manipulations on strings. These string functions make it easier to perform tasks such as string copy, concatenation, comparison, length, etc. The <string.h> header file contains these string functions. T
7 min read
Constants in C++
Constants in C++ refer to variables with fixed values that cannot be changed. Once they are defined in the program, they remain constant throughout the execution of the program. They can be of any available data type in C++ such as int, char, string, etc. Let's take a look at an example: [GFGTABS] C
4 min read
String Tokenization in C
In C, tokenization is the process of breaking the string into smaller parts using delimiters (characters treated as separators) like space, commas, a specific character, or even a string. Those smaller parts are called tokens where each token is a substring of the original string separated by the de
3 min read
Tokenizing a string in C++
Tokenizing a string denotes splitting a string with respect to some delimiter(s). There are many ways to tokenize a string. In this article four of them are explained: Using stringstreamA stringstream associates a string object with a stream allowing you to read from the string as if it were a strea
4 min read