API disabled? #4
Comments
|
Yes, @SoulGit, I see the same thing! In fact I was just looking at the problem. |
|
Btw, the push from yesterday should have been very benign. I bumped the required version of yard to accommodate a security update. Locally, I hadn't changed it - just in github, and my local sunpower script is failing anyway. So I don't think the yard update is to blame. |
|
Apologies my message wasn't clear. I'm sure your commit had nothing to do with the issue - in fact - I just used the URLs from your repo as a launch point to create my own python plugin for Indigo Domotics about two years ago. I starting receiving errors last night (8/12/19 ~7:30pm Pacific) when my prior auth was invalidated (and I had plenty of time left on the session). The main portal does mention a maintenance window from 6 to 8am Pacific today, but that seems to have come and gone. From what I can see, they moved the service over to AWS Cloudfront and the endpoint is no longer configured to accept POST requests necessary for auth. Time to get out my Saint Bernard and figure out what is going on . . . |
|
Looks to be a new auth URL: https://elhapi.edp.sunpower.com/v1/elh/authenticate |
This is a partial response to issue #4. The API changed dramatically recently.
|
Looking good Jeff. I also spent time looking through the XHRs and have most of my plugin working again. Where did you find the /components endpoint? I couldn't seem to find that one and what I am actually looking for is the replacement for the old getACPVModuleInfo. That endpoint listed moduleID, tilt, XY position and other useful info. That's the last piece I need to get working again . . . Let me know if you've come across anything like that in your research. Thanks! |
|
@SoulGit, I found the /components endpoint in app.min.js, injected by the dashboard at https://monitor.us.sunpower.com/#!/dashboard I have been really disappointed in two things in the new API:
As to the fields from getACPVModuleInfo - I don't see them in /components. I think those data may be supplied by a different API still accessible from the "partner" Flash-based site at https://www.sunpowermonitor.com/partner/partner.aspx# (use your usual username/password). Thanks for your interest and help. |
|
Ah - I missed looking through app.min. Wish there was more in there, but seems ACPV is no longer available. I was reading around and saw a post that the individual panel data leads to additional support calls as folks tend to believe something is awry when all panels are not producing similarly. However, I never saw individual panel data in the old portal, and the average user is not consuming from the API, so I don't see why it needed to be removed or obfuscated in the new API. I agree with you Jeff that the approach and the new API is unacceptable. There are folks who simply like to collect data on a system they invested in. I have no interest to contact SunPower to let them know panel 8 is producing four watts less than panel 10. I can however use this data to analyze trends over time and know which trees need trimming on my property. I just got my plugin working again yesterday and was so pleased that I didn't actually analyze any data. Sure enough, you are correct, the reported values are way off. The sun has been shining brightly over here for two hours and I still see 0 for CurrentProduction. Unacceptable and useless as there is no way to tie this data to other metrics (such as a rainforest consumption unit), especially considering there is no timestamp :( In your case to SunPower, do you mentioned that you are using the API directly or rather not seeing data in their portal? I was under the impression that direct use of the API was not supported, but if that is not the case then I think we can definitely run this up the chain. I'd be happy to use the partner API if that provides the necessary data, although from a quick review there isn't much in the way of XHR requests as it's all happening from a VERY SLOW flash applet. BTW - my plugin enabled the use of the old Logout functionality. I have adapted this call to the new endpoint and didn't see it in your updates. If you send a DELETE request to /session with your token in the SP-CUSTOM header, it will log you out. |
|
Ugh - they also touted the new portal to be faster and more reliable than the old. I am now seeing extensive delays in response with many timeouts . . . |
|
@SoulGit, thank you for your continued interest. Thank you too for informing me about logout method. I'll add that to my notes. |
|
Looks like recent change: https://monitor.us.sunpower.com/v09272019065330/envcfg.js (function (w) {
w.__env = {
SP_ELH_VERSION: 'SP_ELH_VERSION_2', // SP_ELH_VERSION must match SP_ELH_VERSION lambda variable in ElhApi cloudformation
ELH_API_USER: w.__envapidom + 'v1/elh/user/',
ELH_API_ADDRESS: w.__envapidom + 'v1/elh/address/',
ELH_API_ASYNC_RESULT: w.__envapidom + 'v1/elh/asyncResult',
ELH_API_PASSWORD_RESET: w.__envapidom + 'v1/elh/passwordreset/',
ELH_API_AUTHENTICATE: w.__envapidom + 'v1/elh/authenticate',
ELH_API_SESSION: w.__envapidom + 'v1/elh/session',
ELH_API_AUTHENTICATEKIOSK: w.__envapidom + 'v1/elh/authenticatekiosk',
ELH_API_FEEDBACK: w.__envapidom + 'v1/elh/processFeedback',
ELH_API_CHECK_EMAIL: w.__envapidom + 'v1/elh/register/checkEmail',
ELP_API_VALIDATE_SN: w.__envapidom + 'v1/elh/register/validateSerial/',
ELH_API_CREATE_ACCOUNT: w.__envapidom + 'v1/elh/register/createAccount',
ELH_API_SEND_ACCOUNT_NOTIFICATION: w.__envapidom + 'v1/elh/register/sendAccountNotification',
ELH_API_VALIDATE_EMAIL: w.__envapidom + 'v1/elh/register/validateEmail',
ELH_API_COLLECTOR: w.__envapidom + 'v1/elh/wifi/collector',
ELH_API_DEVICE_STATUS: w.__envapidom + 'v1/elh/wifi/status',
ELH_API_WIFI_NETWORKS: w.__envapidom + 'v1/elh/wifi/networks',
ELH_API_SET_WIFI_NETWORK: w.__envapidom + 'v1/elh/wifi/setNetwork',
ELH_API_SET_WIFI_NETWORK_STATUS: w.__envapidom + 'v1/elh/wifi/setNetwork/status',
ELH_API_TOU: w.__envapidom + 'v1/elh/tou',
ELH_API_SET_TOU: w.__envapidom + 'v1/elh/setTou',
ELH_API_REFERRAL: w.__envapidom + 'v1/elh/referral'
};
}(this));SP_ELH_VERSION changed from |
Did you inspect |
|
Thanks @sashkab - I missed your first response a week ago. Yes, it does appear the API has changed. Under v1 I did find the ACPV components (https://elhapi.edp.sunpower.com/v1/elh/address/{{ addressId }}/components?async=false), which created a JSON device discovery report with all of the components of the system (however the serial numbers for individual panels were not provided, nor was the power output.) Now, when I run the same query, the response only contains a single entry for the PV Supervisor." Do you have a working URL that provides ACPV info? |
|
My records show that I saw it using /address/id/components endpoint, but no longer can see it there. |
|
Yes, looks like they have removed it. |
|
I guess, we should try to utilize information from the comment 84 without querying API, but instead connecting to PVS directly. |
|
The truly disappointing part is that the PVS sends useful, timestamped data
to Sunpower, where it is garbled and rendered meaningless, yet accessible
by the version 2 api.
I mean the result is not useful to us - how could it possibly by useful to
Sunpower? It just feels like a total failure. Sorry to vent...
…On Tue, Oct 8, 2019 at 11:58 AM Aleks Bunin ***@***.***> wrote:
I guess, we should try to utilize information from the comment 84
<https://www.solarpaneltalk.com/forum/solar-panels-for-home/solar-panel-system-equipment/19587-mirroring-intercepting-sunpower-monitoring-traffic/page6>
without querying API, but instead connecting to PVS directly.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#4?email_source=notifications&email_token=AAAN4MBS7SEITRR4WYGWLEDQNTJ7BA5CNFSM4ILYRPJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAVHSXQ#issuecomment-539654494>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAN4MFF7HP6OTKT6OEQVXTQNTJ7BANCNFSM4ILYRPJA>
.
|
|
PVS sends data to them; they store and aggregate and provide it via non-public API which is all they want to share with their customers. Another thing I found, that they grab and store all wifi access points information available nearby. What else they capture? |
|
Btw, does anyone understands what |
|
New endpoint: |
|
Is there a way to test this in Postman? Trying to follow this thread a little bit. and a little confused on which URL to follow. https://monitor.us.sunpower.com/v2/elh/authenticate using this url i at least get a response back saying access denied. using the url everybody talking about on this thread: https://elhapi.edp.sunpower.com/v1/elh/authenticate i'm getting "Endpoint has not been found, Code EDP4041" Since the first url seems to get me further, what is the authentication parameters or correct way to do the authorization via Postman if anybody has tried that? |
Because you're sending GET instead of POST. |
Which url were you referring to? Iv tried both POST AND GET. The POST is whats giving me the "Endpoing has not been found" error. Technically i get the error on GET as well. Which is why i thought maybe i'm doing the params wrong or using the wrong API.
|
|
idk. My code is like this and works. I think you need to say that you send data as "'application/json" and accept "'application/json". import requests
headers = {'Content-Type': 'application/json',
'accept': 'application/json',
'User-Agent': 'Mozilla/1.22 (compatible; MSIE 2.0; Windows 3.1)'}
data = {"username": "[email protected]",
"password": "Passw0rd",
"isPersistent": False}
r = requests.post("https://elhapi.edp.sunpower.com/v1/elh/authenticate", headers=headers, json=data)
r.raise_for_status()
j = r.json()
print(j) |
|
Out of curiosity, how many of you have "Communication Issue" message in the us.monitor.sunpower.com? |
|
Right now, I see the banner "Possible error in your displayed data. Learn
more."
Clicking the link for more information, I get
"
Temporary error in your displayed data
------------------------------
Your data may not be displaying properly right now. Don't worry, your
system performance is unaffected. We are working on getting this fixed as
soon as we can.
------------------------------
"
That said, when I poll the API, I am getting reasonable results.
…On Tue, Jan 21, 2020 at 10:20 AM Aleks Bunin ***@***.***> wrote:
Out of curiosity, how many of you have "Communication Issue" message in
the us.monitor.sunpower.com?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#4?email_source=notifications&email_token=AAAN4MCQ2POXXO6BXBSFVK3Q644FPA5CNFSM4ILYRPJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJQXKNA#issuecomment-576812340>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAN4MGYCZOAISSYJY6BJMTQ644FPANCNFSM4ILYRPJA>
.
|
|
I am seeing the same as Jeff. I have all but given up on their monitoring systems and system as a whole. I don't even receive their automated "Your Latest SunPower Monitoring Monthly Report" anymore. Last one that came out was end of October. Very disappointed where things have gone with Sunpower. I am going to be expanding my capacity shortly and I'll be looking to replace their system entirely as I have no idea how it is performing day in and day out. |
I don't either. But... it seems that checkbox was checked off when I logged into the mine account over the weekend. I checked it on and will see if I'll get it in early February. On other hand "partner" site which uses flash, can generate these reports for any specific date ranges. The temporary error been there for a month or so, but it never was causing data not showing up. Now it's "communication error" and PVS rebooting every 20-25 minutes. Which means something is wrong either with PVS, or with their online services. If later, it's easy. But what I've seen it's issue with PVS and/or software on PVS. |
|
Sure enough mine was turned off as well. Nice that they made that change without notifying end users . . . |
|
Mine got fixed sometime today. Seems like firmware upgrade on PVS happened around noon. |
|
Hi Jeff, Any of their api gives a panel level data? |
|
The API does not, as far as I know, give access to panel-level data. In
fact, it appears to be a digested summary of data, sourced from a database
at sunpower's service. It is not actually communicating directly with your
system.
There is another communication channel from your supervisor hardware to the
sunpower service. Panel status is available there, but I do not know the
API.
…On Mon, Apr 13, 2020 at 11:30 AM shelly varghese ***@***.***> wrote:
Hi Jeff,
Any of their api gives a panel level data?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#4 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAN4MB76PJCQI6JLZRHQLDRMNK33ANCNFSM4ILYRPJA>
.
|
|
I doubt there is an API -- PVS pushes data at schedule, about once every 5 minutes or so, but channel is encrypted so unappeasable. I wrote a small script which parses PVS status page into several csv files using Raspberry Pi connected to the "service" ethernet port of the PVS. |
Would you be willing to share your Pi image that does so, and perhaps a writeup on necessary changes needed to configure this at another location? |
|
@anthonysparrow I created a different repo to document the hardware I use (a raspberry pi zero w and a ethernet hat, which fit into the Supervisor box itself). I also wrote a small Ruby app that grabs the data and puts it into Influx for graphing with Graphana. Hope this helps. |
I've used standard raspbian and tiny python script. You need python 3.6 or newer for this to run, and import requests
from datetime import datetime
import time
import json
from pathlib import Path
import csv
PVS5_URL = "http://172.27.153.1/cgi-bin/dl_cgi?Command=DeviceList"
FIELD_NAMES = {
"pvs": [
"DETAIL",
"STATE",
"STATEDESCR",
"SERIAL",
"MODEL",
"HWVER",
"SWVER",
"DEVICE_TYPE",
"DATATIME",
"dl_err_count",
"dl_comm_err",
"dl_skipped_scans",
"dl_scan_time",
"dl_untransmitted",
"dl_uptime",
"dl_cpu_load",
"dl_mem_used",
"dl_flash_avail",
"panid",
"CURTIME",
],
"inverter": [
"ISDETAIL",
"SERIAL",
"TYPE",
"STATE",
"STATEDESCR",
"MODEL",
"DESCR",
"DEVICE_TYPE",
"SWVER",
"PORT",
"MOD_SN",
"NMPLT_SKU",
"DATATIME",
"ltea_3phsum_kwh",
"p_3phsum_kw",
"vln_3phavg_v",
"i_3phsum_a",
"p_mpptsum_kw",
"v_mppt1_v",
"i_mppt1_a",
"t_htsnk_degc",
"freq_hz",
"stat_ind",
"origin",
"OPERATION",
"CURTIME",
],
"powermeter": [
"ISDETAIL",
"SERIAL",
"TYPE",
"STATE",
"STATEDESCR",
"MODEL",
"DESCR",
"DEVICE_TYPE",
"SWVER",
"PORT",
"DATATIME",
"ct_scl_fctr",
"net_ltea_3phsum_kwh",
"p_3phsum_kw",
"q_3phsum_kvar",
"s_3phsum_kva",
"tot_pf_rto",
"freq_hz",
"CAL0",
"origin",
"OPERATION",
"CURTIME",
],
}
def get_usage():
"""Get Information from PVS5"""
try:
r = requests.get(PVS5_URL, timeout=10)
except requests.exceptions.ConnectionError as err:
print(f"{datetime.now():%Y.%m.%d %H:%M:%S} {err}")
return
js = r.json()
dte = datetime.now()
data = {}
if js.get("result") == "succeed":
data = {}
for device in js.get("devices"):
device_type = "".join(device.get("DEVICE_TYPE").split()).lower()
data.setdefault(device_type, []).append(device)
else:
print(f"{dte:%Y.%m.%d %H:%M:%S} result={js['result']}")
return
for key in data.keys():
dump_file = Path(f"{dte:%Y%m%d}_{key}.csv")
write_header = not dump_file.exists()
with dump_file.open("a+", newline="") as csv_file:
writer = csv.DictWriter(csv_file, fieldnames=FIELD_NAMES.get(key))
if write_header:
writer.writeheader()
for v in data.get(key):
writer.writerow(v)
if __name__ == "__main__":
while True:
try:
get_usage()
except Exception as err:
print(f"Unhandled Exception {err}")
print(f"{datetime.now()}", end="\r")
time.sleep(60) |
|
@sashkab Looks nice! I saw another post where they just set up HAProxy (although any would do) to proxy the LAN API via WiFi. Seems pretty straightforward and allows you to do the API logic on a more capable machine that's easier to access. |
|
@arcreative |
|
@jeffkowalski Nice! Instead of going with Node-RED, I did something a bit more simple that I can run along with my other docker-compose services. It's a ruby script that you can build into a Docker image for easy running without having to install ruby on the host: https://gist.github.com/arcreative/baf7174d7c5574a515c5cf4bb5877bbb |
|
@arcreative your script looks simple enough, but you're ignoring error state on inverters, I believe. When inverters doesn't produce electricity (or after PVS restart and before first reported value), they will report inverters = data['devices'].filter { |i| i['DEVICE_TYPE'] == 'Inverter' }.filter { |i| i['STATUS'] != 'error' }I'm not sure (yet) if power meter can have an error state... so you might want to look out for that too. An yes, HAProxy is simple enough to configure so you could access PVE data from another host. |
|
@sashkab Thanks! I didn't really dig into it too too much as you seem to have noticed from my complete lack of error handling :-). That's good info, I'll make it more robust when I get a little time. I'm still having trouble on the Home Assistant side figuring out exactly what statistics to use and how to do them, so there's still a bit of work ahead. |
|
@jeffkowalski Looks like the endpoint https://elhapi.edp.sunpower.com/v1/elh/authenticate no longer exists and responds with a 401. Any ideas on if this script will still work with some minor tweaks? |
Hi Jeff -
Is this API still available for you? I've been using it for about two years and just yesterday I am now getting access denied and with a new hostID field being returned in the the JSON. Coincidentally I see you pushed an update to the repo 13 hours ago - so wondering if you are seeing the same thing that I am. The website also no longer returns any REST info . . . at least for me. Thanks!
The text was updated successfully, but these errors were encountered: