Very much smart people (piefedimages.s3.eu-central-003.backblazeb2.com)
 
Post:"this is who you're arguing with online" Picture of a neural network. Reply:"there's too many people with "AI/ML" in their bio asking what this image is"
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (3 children)

All seems pretty random, and not very scientific. Why not try 5 layers, or 50, 500? A million nodes? It's just a bit arbitrary.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 year ago

    It is random, at least while it is learning. It would have most likely tried 5 layers, or even 50.

    But the point is to simplify it enough while still working the way it should. And when maximizing the efficiency, you generally get only a handful of efficient ways your problem can be solved.

  • source
  • parent
  • [–] 1 point 1 year ago (1 child)

    In practice it's very systematic for small networks. You perform a search over a range of values until you find what works. We know the optimisation gets harder the deeper a network is so you probably won't go over 3 hidden layers on tabular data (although if you really care about performance on tabular data you would use something that wasn't a neural network).

    But yes, fundamentally, it's arbitrary. For each dataset a different architecture might work better, and no one has a good strategy for picking it.

  • source
  • parent
  • hideshow 1 child comment