Very much smart people (piefedimages.s3.eu-central-003.backblazeb2.com)
 
Post:"this is who you're arguing with online" Picture of a neural network. Reply:"there's too many people with "AI/ML" in their bio asking what this image is"
you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 year ago (4 children)

Probably because there's no good reason.

At least one intermediate layer is needed to make it expressive enough to fit any data, but if you make it wide enough (increasing the blobs) you don't need more layers.

At that point you then start tuning it /adjusting the number of layers and how wide they are until it works well on data it's not seen before.

At the end, you're just like "huh I guess two hidden layers with a width of 6 was enough."

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 1 year ago (3 children)

    All seems pretty random, and not very scientific. Why not try 5 layers, or 50, 500? A million nodes? It's just a bit arbitrary.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 year ago

    It is random, at least while it is learning. It would have most likely tried 5 layers, or even 50.

    But the point is to simplify it enough while still working the way it should. And when maximizing the efficiency, you generally get only a handful of efficient ways your problem can be solved.

  • source
  • parent
  • [–] 1 point 1 year ago (1 child)

    In practice it's very systematic for small networks. You perform a search over a range of values until you find what works. We know the optimisation gets harder the deeper a network is so you probably won't go over 3 hidden layers on tabular data (although if you really care about performance on tabular data you would use something that wasn't a neural network).

    But yes, fundamentally, it's arbitrary. For each dataset a different architecture might work better, and no one has a good strategy for picking it.

  • source
  • parent
  • hideshow 1 child comment