Yes... The simple explanation was some wavelengths are absorbed and others are reflected; we see the reflected visible light as color. But going a bit deeper, its not simple as the wavelength. What you perceive goes much more beyond than the wavelengths of light. The brain has evolved to perform lots of context-dependent corrections to correct for the colour and intensity of the ambient light.
As you said the quantum explanation isn't that simple at all. It's because duality nature (wave-particle) and what governs it is the fundamental conservation of energy principle.
Electrons and nuclei have a finite mass and when light is tugging the reduction of light energy gets transferred to matter - and this is
absorption. The converse also happens, the motion of charged particles in matter can give up their energy to create light
emission. In either case the conservation of energy holds true.
Take for example the simple Hydrogen atom - a single proton and an orbiting electron. In the normal state it's at the lowest energy level. When the H atom absorbs light, the electron jumps one lever or more depending on the energy absorbed. To jump a level it needs exactly a discrete amount of energy - no more, no less. This energy
corresponds to the wavelength of light that it absorbs. The electrons absorb
only the photons that give them exactly the right energy they need to jump levels.
In the visible part of the spectrum, hydrogen absorbs light with wavelengths of 410 nm (violet), 434 nm (blue), 486 nm (blue-green), and 656 nm (red). Each of the absorption lines corresponds to a specific electron jump. The shortest wavelength/highest energy light (violet 410 nm) causes the electron to jump up four levels, while the longest wavelength/lowest energy light (red 656 nm) causes a jump of only one level.
The opposite also happens. Electrons can also lose energy and drop down to lower energy levels. When this happens it emits photons with the same amount of energy—the same wavelength—that it would need to absorb in order to move up between those same levels. This is why hydrogen’s emission spectrum is the inverse of its absorption spectrum as shown below.
Images - NASA.
PS: Einstein wasn't awarded the Nobel Prize for his paper on Relativity (he should have). But he won it for his paper on
Photoelectric Effect. It's worth noting the following comment by him at his later stages in life.
"
All the fifty years of conscious brooding have brought me no closer to answer the question, “What are light quanta?” Of course today every rascal thinks he knows the answer, but he is deluding himself."