Skip to content

ESQL: document_found isn't always the number of "documents" foundΒ #131771

@nik9000

Description

@nik9000

Description

I added documents_found and values_loaded back in #127161 to give users a rough sense of how much work the engine is doing. It's been super useful! In #127355 I used it to show that we were pushing a query to lucene.

But, confusingly, it isn't always the number of "documents" "found". If you do a query like:

FROM my-index,*:my-index
| STATS c=count(*)

You can get:

{
  ...
  "documents_found": 3,
  "values_loaded": 0,
  "values": [[4]]
}

values_loaded is accurate here - we didn't load any values to count all the rows. Yay.

But documents_found is bizarre here. 3?! There are 4 documents in the index. The result says so. What gives?

Well, I took "give users a rough sense of how much work the engine is doing" too far and made things confusing. See, the thing that makes the count in this case emitted total counts. Probably 1, 1, and 2. I though "those are like documents", let's count them as a "document" "found". They aren't "found", they are read from metadata. They aren't "documents". They are a count I read from metadata.

If this field were called "engine_work" they I did is right. But I didn't call it that.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions