856
We put the Thing That Can't Do Numbers™ in your spreadsheets (programming.dev)
submitted 1 year ago by to c/iiiiiiitttttttttttt@programming.dev
96 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 35 points 1 year ago (16 children)

I'm amazed that it can't figure out that it should just defer arithmetic to like, all the functions that Excel already has, rather than trying to LLM its way to the correct answer.

If you could ask it to do something in Excel that you don't know how to do, and it did it using Excel functions and maybe explained it to you, that would be useful, but as it is now, that thing isn't trustworthy at all.

  • permalink
  • fedilink
  • source
  • hideshow 16 child comments
    •
  • [–] 19 points 1 year ago (14 children)

    They straight up tell you to not use it for math. It's for analyzing a bunch of text you shoved into a spreadsheet, say for example customer testimonials or something. Making it work most of the time would actually be worse, because then people would be more inclined to use it instead of writing the formulas to do it right.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 14 child comments
    •
  • [–] 6 points 1 year ago (12 children)

    Shouldn't excelAI know when it's doing math?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] 6 points 1 year ago (11 children)

    No AI truly knows what it's doing. You can give it things to call out to, but you can't know for sure if it will use them, and definitely can't know it will pass it the right parameters.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 1 point 1 year ago* (10 children)

    It can recognise the difference between text(>63) and numbers (<58). Even if it's just a preprocessing screening.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 1 point 1 year ago (9 children)

    How is looking at ascii values supposed to help when someone prompts it with "calculate the sum of the numbers above"? The whole point is that no matter what kind of prescreening you add to an LLM, people will write prompts which are missed by the screening.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 0 points 1 year ago (8 children)

    How is looking at ascii values supposed to help when someone prompts it with "calculate the sum of the numbers above"?

    Because you can check if values input from the spreadsheet are non-numeric.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 3 points 1 year ago (7 children)

    There are no values from the spreadsheet in this case. "The numbers above" are just text to the LLM.

    They could of course require that optional cell or cell range parameter after the prompt, but that would eliminate some use cases. "Generate some text", one of the stated use cases in the help text, doesn't reference any cells.

    Also, numbers in Excel aren't necessarily as clear cut as you make it seem. Excel famously thinks everything is a date, and how number-y must a number be before it isn't okay?

    Not to mention there are other things to do with numbers which don't require arithmetic. What if someone wants to have Excel translate 34 to "thirty-four"? Or have Excel generate a poem 34 words long? Or whatever else nonsense people might try.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 1 year ago (6 children)

    There are no values from the spreadsheet in this case.

    What hill are you trying to die on here? In the picture, the numbers input are clearly 1,2 and 3.

    They could of course require that optional cell or cell range parameter after the prompt,

    No. It is trivial to identify if the input is text or value. You don't need an LMM to do that.

    Excel famously thinks everything is a date

    That's only interpreting data entry. Date values are stored in cells as doubles.

    Not to mention there are other things to do with numbers which don't require arithmetic.

    Yes these are the use cases where an LLM would add some (questionable) value.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 1 year ago* (last edited 1 year ago) (5 children)

    The text field says =COPILOT("sum the numbers above"). It doesn't work that way. Excel does not have any concept of what "above" means here. Those numbers are not used in the calculation whatsoever. To reference those numbers, the field should say =COPILOT("sum the numbers in", A1:A3).

    What the user did here was ask the LLM to generate some text based on a text prompt and no other data, and the LLM decided to answer with a string containing only digits.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -1 points 1 year ago (4 children)

    Excel does not have any concept of what "above" means here.

    Excel knows the address of the calling function.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 1 year ago (3 children)

    Yes, but "above" just goes into the LLM, which does who-knows-what with it, and certainly isn't designed to address cells that way. So to the LLM that's just like any other arbitrary text.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] -1 points 1 year ago (2 children)

    Yes, but "above" just goes into the LLM,

    I don't believe even Microsoft are this stupid. The llm will have tools to interrogate the spreadsheet.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 1 year ago

    There would be no way to do that reliably. There's too much weird stuff people might say to reference things, and the LLM would definitely act on the wrong cells more often than not.

    Excel already has a perfectly unambiguous way to provide a specific range of cells, Which is why the =COPILOT() function lets you supply those in the second parameter. I'm assuming they get passed to the LLM as context, likely encoded as a markdown table. LLMs love parsing markdown, apparently.

    The user provided no such range of cells, though, so the LLM is most likely seeing none of those other cells, and is just working based on random values from the Internet.

  • permalink
  • fedilink
  • source
  • parent