Only first page of articles in The Economist

[kovidgoyal]

I guess I can try to see if implementing the login using a headless js browser and then copying the cookies will work, but so many moving parts...

[jwhittin]

Oh dear, sounds fragile. But if you could, that would be really great. Happy to test it you need it.

Many thanks.

[unkn0wn]

this is just temporary. i dont know for how long this can work.. but it works for me, for this weeks edition.

[RobbieW]

Originally Posted by unkn0wn

this is just temporary. i dont know for how long this can work.. but it works for me, for this weeks edition.

I was missing pages in the UK section, this variant fixed that just now - thank you

[jwhittin]

Originally Posted by unkn0wn

this is just temporary. i dont know for how long this can work.. but it works for me, for this weeks edition.

It works for me too. Many thanks unkn0wn.

I note that it produces a much smaller file than the normal recipe. What's the difference? (I've tried but can't really understand the recipe script) Also, FYI - it only seems to work for the current edition - I tried putting in last week's date on line 18 but it still brought up the current edition.

[kovidgoyal]

@unkn0wn: Nice find. I have updated the builtin recipes to use the GraphQL query to load the content, let's see if it continues to work next week. I have kept the old code around so if it does break next week we can still re-use it easily.

This, of course breaks downloading past editions, but I imagine to re-enable that we need to find the correct content_id give an edition date.

[jwhittin]

Originally Posted by kovidgoyal

I have updated the builtin recipes to use the GraphQL query to load the content

Just tested and works well for me. Many thanks Kovid.

[unkn0wn]

Originally Posted by kovidgoyal

I have kept the old code around so if it does break next week we can still re-use it easily.

This, of course breaks downloading past editions, but I imagine to re-enable that we need to find the correct content_id give an edition date.

That's why I didn't want to push this update. I was not sure if the link would work next week or even tomorrow and didn't want to lose old code.

i think there's a different query for past editions.. i'll try to find it, if this works for next weeks edition.

[Steven630]

Originally Posted by unkn0wn

this is just temporary. i dont know for how long this can work.. but it works for me, for this weeks edition.

Hello. Economist Espresso recipe, which was a simplified version of Economist recipe, started to fetch only the first two paragraphs since yesterday. Would you please take a look? Thank you.

https://www.economist.com/the-world-in-brief

[unkn0wn]

https://github.com/kovidgoyal/calibr...c1e68b3ed201b0

[Steven630]

Originally Posted by unkn0wn

https://github.com/kovidgoyal/calibr...c1e68b3ed201b0

Thank you.

There is a little bug with Economist recipe. The flytitle appears on a separate page before the h1 title(thus before the rest of the article) on my Kindle. Flytitle, h1 title, and rubric should be on the same page. I don't know why this happens.

[unkn0wn]

Originally Posted by Steven630

Thank you.

There is a little bug with Economist recipe. The flytitle appears on a separate page before the h1 title(thus before the rest of the article) on my Kindle. Flytitle, h1 title, and rubric should be on the same page. I don't know why this happens.

idk maybe lower the font size and check if it re-adjusts.

[Steven630]

Originally Posted by unkn0wn

idk maybe lower the font size and check if it re-adjusts.

There's a page before the article with only flytitle followed by blank content (the entire page has only flytitle). The font size is small enough

[Steven630]

I ended up adding the text of flytitle to h1:

        if 'flyTitle' in data and data['flyTitle']:
            title_text = f"{data['flyTitle']}: {data['title']}"
        else:
            title_text = data['title']
        #E(article, 'div', data['flyTitle'] , style='color: red; font-size:small; font-weight:bold;')
        E(article, 'h1', title_text, title=safe_dict(data, "url", "canonical") or '')

[RobbieW]

An observation - the reworked recipe seems to download the complete paper, thank you. However the filesize is now much smaller than previous. The current copy is c. 7Mb, its been 20-30Mb for some while - is this expected ?