Is there a way to download or otherwise automatically access my story statistics page in a programmatic way? I'm not asking for access to anything more than what's already there, but I'd like to be able to look at the data over a period of time. And I know you already have trouble with web scrapers etc.
If you're talking programmatic I assume you mean using a programming language. If so, just get the page and extract the data. If your programming language can't do that use wget to get the page and extract the data from that. You'll have to read up on it to achieve that because SOL assumes you are logged in when you get that page. Using something like wget is not scraping, it's just like a browser that gets the page so no difference for SOL.
@Keet ... using wget or writing a piece of code to do that is actually exactly what "scraping" is :D
I know how to write that code, but it's both non-trivial to do so and also I'm pretty sure Lazeez would prefer I didn't, which is why I asked the question here.
If you're not going to get the data yourself the only alternative is that Lazeez offers special links (API) to serve the data. A little like news feeds but on request and user specific. I can think of a few I would want for the ReaderInfo site ;)
If you're talking programmatic I assume you mean using a programming language.
Or downloading the data to a spreadsheet.
Or downloading the data to a spreadsheet.
That's the problem: you don't get data, you get a html page from which you have to extract the data you want. I do it every year when the results from the Clitoride awards are published. Download the html, extract the data and dump it in a mysql database. From there the ReaderInfo site can display awards for the contributor pages. But as said, you get a html page and you have to get the actual data from that page, usually with some programming code so you can do it over and over again.
ETA: that's what the initial question was about: a way to get structured data, not html.
That's the problem: you don't get data, you get a html page from which you have to extract the data you want.
I was thinking of what Amazon KDP does for sales figures.
There's a button called "Generate Report" (the "what is this?" says: Click this button to generate a detailed report containing orders, sales and royalties earned for the selected time period).
When I click on the button I get something downloaded. I then double click on that file and end up with an Excel spreadsheet with all the data. (I actually copy data from that spreadsheet to my permanent sales spreadsheet.)
I wouldn't use it for my SOL stats simply because I don't pay much attention to those, but for someone like @Charles Jeffries who wants those numbers, it's an option. Is it worth Lazeez's time? I was simply giving the option.
When I click on the button I get something downloaded. I then double click on that file and end up with an Excel spreadsheet with all the data. (I actually copy data from that spreadsheet to my permanent sales spreadsheet.)
What you get from KDP is probably a CSV file (Character Separated Values). Any spreadsheet program can import such a file and put the data into rows and columns. It's extremely simple to generate a CSV file from a database. Just create the query and export it as a CSV file. Takes less resources than generating a html page :)
It's extremely simple to generate a CSV file from a database. Just create the query and export it as a CSV file. Takes less resources than generating a html page
I didn't know what it was called. Or even how it worked (all I knew was that I ended up with a spreadsheet with the data). But I thought it was what the requester was asking for.
I didn't know what it was called. Or even how it worked (all I knew was that I ended up with a spreadsheet with the data).
With most default OS installs if you double click on a CSV file it is opened in a spreadsheet program. Create a plain text file with
"A","B","C",1,2,3
"D","E","F",4,5,6
and save it as test.csv
Now double click it and it will open in Excel with two rows with each six columns :)
Try opening the file you get from amazon with a text editor.
Character Separated Values
Are you sure it doesn't mean Comma Separated Values in this instance?
AJ
Are you sure it doesn't mean Comma Separated Values in this instance?
Also correct but the more generic name is Character Separated Value because the field separator can be other characters too. In my work I usually use a pipe (|) to avoid problems with commas inside a field value.
Also correct but the more generic name is Character Separated Value because the field separator can be other characters too. In my work I usually use a pipe (|) to avoid problems with commas inside a field value.
That may be technically true, but Excel and other spreadsheet applications will only handle comma separators without going through an explicit import process and any spreadsheet application that sees a .csv extension is going to expect comma separators.
That may be technically true, but Excel and other spreadsheet applications will only handle comma separators without going through an explicit import process and any spreadsheet application that sees a .csv extension is going to expect comma separators.
Commas are the default but you can still get the dialog if the line separator is not the default. For example Windows and Linux don't have the same default for this so exchanging files between them might cause the dialog to show. And what's wrong with an import dialog? In most cases the spreadsheet has already figured out what the separators are so it's just a click on 'OK'. CSV existed before spreadsheets so it's not like spreadsheets command what a CSV should look like. It's just a really easy format to import. My use of CSV is rarely through a spreadsheet program so it's very convenient to avoid 'wrong' commas in field values by using a different separator.
And what's wrong with an import dialog?
Nothing, but in Windows systems. If you look at the file type filter list or the file type selection it explicitly lists .csv as comma separated values. There is a separate list for plain text as .txt and one for tab delimited also as .txt.
And what's wrong with an import dialog?
Nothing, but in Windows systems. If you look at the file type filter list or the file type selection it explicitly lists .csv as comma separated values. There is a separate list for plain text as .txt and one for tab delimited also as .txt.
https://fileinfo.com/extension/csv
What is a CSV file?
A CSV file is a comma-separated values file commonly used by spreadsheet programs such as Microsoft Excel or OpenOffice Calc. It contains plain text data sets separated by commas, with each new line in the CSV file representing a new database row and each database row consisting of one or more fields separated by a comma.
CSV existed before spreadsheets
I was certainly moving data around in CSV (comma separated value) format before I encountered my first spreadsheet. It was one of the file formats supported by Microsoft Basic acting as the home computer operating system.
AJ
It's not too hard to create a CSV.
Which fields would you be interested in downloading?
Sure: a CSV of the stats page would be great. Although as I was hinting at, an API to download it would be even better, so I didn't have to click the button every day (or week or month or whatever).
I'm interested in most of it: #Comments, Total Downloads, #Votes, Score
Done. I added a 'Download CSV' link to the stats page. The link has a key embedded so it can be used without logging in to pull the data.
I gave it a try. There's an issue with the naming of the download file.
If I go straight for save file, it gets saved as showStats.php
If I go for open file it downloads as showStats.php.csv
It ought to be consistent either way and the file name should probably be showStats.csv
There's an issue with the naming of the download file.
Heh, I did all my testing using curl, I didn't think of naming the file at all.
I fixed that. Now it's 'pen-name-date-stats.csv' for example:
anonymous-2022-02-05-stats.csv
so it can be used without logging in to pull the data.
Doesn't that mean that anybody who figures out the link can download any authors stats?
Doesn't that mean that anybody who figures out the link can download any authors stats?
Yes. All you have to do is figure out the keys. It's a small challenge.
I can provide you a sample if you wish.
😈
Muhahahaha...
Yes. All you have to do is figure out the keys. It's a small challenge.
Ah, I missed the 'embedded key' part. :)
I should have known, basics and such things...
I added a 'Download CSV' link to the stats page. The link has a key embedded so it can be used without logging in to pull the data.
I'm confused. I am always logged on so I don't know what would happen if I tried to access my author's stats page without being logged on, but I assume I can't. How would the system know who I was to give me the correct stats page if I wasn't logged on?
Since the "Download CSV" link is on the author's stats page, and you can only get to the stats page by being logged on (my assumption), how can you even see the "Download CSV" link if you're not logged on?
how can you even see the "Download CSV" link if you're not logged on?
While logged in, you right click on the link and select 'copy'.
After one copies the link you can paste it in a script. The script can get the CSV without having to log in because the copied link has a key that identifies the pen name in question.
