The Asset Intelligence Evaluation Framework: 40 Questions to Ask Before You Buy

We wrote this framework knowing full well that competitors' prospects will use it. Good. The category is young enough that buyers are still evaluating asset intelligence platforms with monitoring-era checklists — screens, alarms, integrations — and the result is expensive shelfware across the industry. These forty questions are the examination we believe every platform in the category should have to pass, and we hold ourselves to every one of them — including publishing our own answers to the hardest ones at the end. Use it as an RFP skeleton, a demo script, or a scorecard. Insist on answers demonstrated against live plant data. Slideware answers every question fluently.

I. Detection — does it find what availability misses? (1–5)

• •Q1. Can you detect a dead string on an inverter that remains fully online? Show me one, live.
• •Q2. What is your detection method — fixed thresholds, statistical baselines, or a physics model of expected generation from measured weather?
• •Q3. What is your typical time from fault onset to detection, and how do you measure it?
• •Q4. How do you detect gradual losses — soiling accumulation, efficiency drift, degradation above assumption?
• •Q5. What is your false-positive rate, and what mechanism suppresses nuisance alarms without hiding real faults?

II. Explainability — can a human audit the machine? (6–10)

• •Q6. For any alert, can I see the exact rule or model logic that fired, the measured value, and the expected value?
• •Q7. Can I access the raw telemetry rows behind any conclusion, in the same interface, without an export?
• •Q8. Is your detection methodology documented and open to challenge by my engineers?
• •Q9. When your AI recommends an action, does it show the evidence chain — deviation, peer comparison, duration, pattern signature?
• •Q10. Can I reproduce any number on any screen from first principles?

III. Loss attribution — is every megawatt-hour accounted for? (11–15)

• •Q11. Can you reconcile theoretical yield from measured weather down to metered export, with every loss bucketed?
• •Q12. Do you separate recoverable, unavoidable, and asset-fault losses — and defend the boundaries?
• •Q13. Is every loss priced in currency, not just energy?
• •Q14. Can attribution be produced for any historical period, or only from installation forward?
• •Q15. How is the expected-generation model validated, and how often is it re-baselined for degradation?

IV. Data & integration — will it survive my fleet's reality? (16–20)

• •Q16. Which inverter OEMs, SCADA systems, and data loggers do you ingest natively, and at what resolution?
• •Q17. How do you normalize a multi-OEM portfolio into one coherent physical model?
• •Q18. What happens to your analytics during telemetry gaps and sensor failures — and do you detect sensor decay itself?
• •Q19. Who owns the data, and in what form can I export all of it — including your derived analytics?
• •Q20. What is onboarding time per plant, measured on your last ten deployments, not your best one?

V. Workflow — does insight become action? (21–25)

• •Q21. Does a recommendation carry effort, time-to-fix, and confidence — or just a description of the problem?
• •Q22. Can related faults be clustered into a single dispatch to cut truck rolls, and can you show me a month's measured savings?
• •Q23. Do work orders flow to my CMMS, or does this become another swivel-chair system?
• •Q24. Are there distinct working views for the executive, the O&M manager, and the technician — built for how each actually works?
• •Q25. Can a technician on site filter to one inverter, one date range, and see exactly which string, which combiner, with the anomaly highlighted in the raw data?

VI. Compliance — will it stand in front of a regulator? (26–30)

• •Q26. Do you compute regulatory performance fields (e.g., NERC GADS) directly from telemetry, with formulas visible?
• •Q27. Is event classification documented with reasoning, and is every human override logged with author and rationale?
• •Q28. Can you produce a complete audit trail for any filed number, months later, as an export?
• •Q29. Do you run the regulator's data-quality validation rules before submission — as a precondition, not an afterthought?
• •Q30. How do you determine contributing operating conditions — with meteorological evidence from the actual event window, or judgment?

VII. Economics — does it speak the owner's language? (31–35)

• •Q31. Show me energy lost this month as a dollar figure, per fault, on the executive's first screen.
• •Q32. Show me O&M savings — avoided dispatches, suppressed false positives, recovered energy — as a monthly ledger.
• •Q33. Can you support P50 gap analysis and weather-corrected trend for investment and lender reporting?
• •Q34. What is your evidence of realized ROI on portfolios comparable to mine?
• •Q35. How does pricing scale — per MW, per site, per module — and what does year three cost, all-in?

VIII. Vendor durability — will this outlast the pilot? (36–40)

• •Q36. How many MW are live on the platform today, across how many OEM types and technologies?
• •Q37. What is your release cadence, and what shipped in the last two quarters?
• •Q38. What happens to my deployment and my data if you are acquired or discontinue the product?
• •Q39. Which references may I call whose portfolio resembles mine — in scale, technology mix, and OEM diversity?
• •Q40. What do your customers use least — and what did you do about it?

Our answers to the five hardest questions

A framework the author won't sit is not a framework, so here is Ellume Vector on the record against the questions vendors most often deflect.

• •Question 1 — the dead string. Yes, live: Vector's string health heatmap renders every string position across every inverter; on a representative demo plant, 240 strings resolve into 187 healthy, 28 warning, 16 critical, and 9 inoperative — on inverters that were 'available' throughout. The critical-string detail view shows the diagnosis written before any human read it: series strings at 6.0 A and 0.1 A, open circuit or combiner fuse failure.
• •Questions 6–10 — explainability. Every event carries four tabs: Summary, Raw Readings (the telemetry snapshot at detection, anomalous rows highlighted with the reason in the row), Rule Logic (the physics rule, measured versus expected, why this rule and not another), and Recommendations (action, effort, time-to-fix, confidence). The full physics documentation is linked from the interface and open to challenge.
• •Question 22 — measured dispatch savings. On one 3 MW fleet plant in one month: five truck rolls avoided, eleven false positives suppressed with zero false dispatches, $3,500 saved — including a cluster of three same-combiner-row inverter faults plus a soiling job resolved in a single visit.
• •Questions 26–30 — compliance. Ellume BRIDGE computes all GADS performance fields from 15-minute telemetry with DRI formulas displayed, classifies events through an AI workflow with ranked cause-code candidates and documented reasoning, routes ambiguity to human reviewers whose every decision and override lands in an exportable audit trail, and runs 29 DRI data-quality validation rules before any submission file is generated.
• •Question 31 — the dollar figure. It is the fifth card on the executive dashboard: Energy Lost, with its currency value — $1,777 against 16,156 kWh on the plant above — because a loss with a price gets managed and a loss without one gets archived.

Frequently Asked Questions

Isn't a vendor-authored RFP self-serving?

Structurally, yes — we wrote the exam we are confident passing. But every question is answerable by any vendor with the underlying capability, and none references a proprietary feature. If the framework tilts the market toward physics, explainability, and audit trails, we are content with that bias.

How long should an evaluation take?

Sixty to ninety days including one live pilot. Longer evaluations without live data are not rigor; they are calendar-shaped indecision.

Should monitoring and intelligence be bought separately?

They are converging, but the classic buying mistake is assuming monitoring-tier tools grow into intelligence. Detection physics and attribution are architectural — analytics retrofitted onto a monitoring stack is where most category disappointment originates.

What is the most commonly skipped question that later hurts most?

Question 18 — sensor decay and telemetry gaps. Every fleet's data is imperfect; platforms that assume clean inputs degrade silently, and the buyer discovers it during the first dispute their data was supposed to settle.

Is there a scoreable version of this framework?

Yes — a weighted, gated scorecard version is available for procurement teams, and Ellume will sit all forty questions on your live telemetry as part of any evaluation.

Sources & Further Reading