It's very different from a Windows PC.
Windows uses an installer database to determine what files are needed and where (source, I did app packaging and deployment for NT4 through Windows 7).
Android: APKs are just zip files that extract to a specific folder - much, much simpler.
When an app is uninstalled the app folder is deleted, though the data folder may be left behind in case the user decides to reinstall the app.
This is the only part that isn't well-defined or well enforced, and why SD Maid is such a useful tool.