I realized that racism isn't limited to physical acts or discrimination

"Believing that there are innate differences between races" is itself considered racism.
Most people think of racism as discriminatory actions taken against groups of people perceived as different races, but that belongs to the most extreme and malicious end of the spectrum.
Simply believing that there are innate differences between races also falls under the category of racism.