Yahoo has upset me

Düber

Well-Known Member
Joined
May 17, 2018
Messages
144
#1
I have a notice board setup at home. It has behaved itself perfectly for a long while now, but yesterday it stopped telling us the news. I told my wife the Government has heard all her comment's and shut us down.
But it turns out I was wrong.

"Important EOL Notice: As of Thursday, Jan. 3, 2019, the YQL service at query.yahooapis.com will be retired. This will impact users of datatables.org as well as developers who creates features using this YQL service. To continue using our free Yahoo Weather APIs, use https://weather-ydn-yql.media.yahoo.com/forecastrss as your new API endpoint. Contact yahoo-weather-ydn-api@oath.com for credentials to onboard to this free Yahoo Weather API service. Other YQL based services that use query.yahooapis.com will no longer operate."

Now that is awesome!
I fear to say that my coding abilities are a little less than what would be bad.
It is a webserver running on a Orange pi with LAMP, with the majority of the code lifted from here
https://github.com/MichMich/MagicMirror
but an older version.

Essentially it is a webpage with weather, a calendar, clock... etc.

The RSS feeds and age of feeds to display are listed in a config.js file and the code below is the relevant section in the main.js file, it then displays the news feed titles at the bottom of the page one at a time from the various feeds.

Searching throughout the day I have found,

https://stackoverflow.com/questions/246577/can-i-serve-rss-in-json
and


as possible solutions.
It is starting to feel like I'm trying to reinvent the wheel with the spokes on the outside.

I don't want this to be a - Try this.....Not working.....try this..., thread, but some advice and guiding in the right approach would be very helpful.


JavaScript:
    // RSS Feed Display.  Updates every 5 minutes.
    function fetchNews() {
        // Yahoow Query Language implementation borrowed from jquery.feedToJSON.js by dboz@airshp.com
        var yqlURL      = 'http://query.yahooapis.com/v1/public/yql';                            // yql itself
        var yqlQS       = '?format=json&callback=?&q=select%20*%20from%20rss%20where%20url%3D';  // yql query string
        var cachebuster = new Date().getTime();                                                    // yql caches feeds, so we change the feed url each time
        var index       = 0;

        // Loop through the feed URLs
        for( var key in feedURLs ) {
            var url = yqlURL + yqlQS + "'" + encodeURIComponent( feedURLs[key] ) + "'" + "&_nocache=" + cachebuster;

            fetchNewsForURL( index++, url );
        }

        // Update again in 5 minutes
        setTimeout( fetchNews, 300000 );
    };

    fetchNews();

    // Actually get a feed.  The function exists just so we can fake passing
    //  extra arguments to getJSON().
    function fetchNewsForURL( index, url )
    {
        $.getJSON( url, function(jsonRSS, textStatus) {
            if( jsonRSS.query.results != null ) {
                // Success; get the list of articles
                var oldestDate = moment().subtract( feedMaxAge.days, "days" ).subtract( feedMaxAge.hours, "hours" );
                var stories = [];
                for (var i in jsonRSS.query.results.item) {
                    // Skip articles older than a certain time
                    //  pubDate string is "Tue, 05 Jul 2016 14:25:29 -0400", so we decode that
                    var storyDate = moment( jsonRSS.query.results.item[i].pubDate, "ddd, DD MMM YYYY HH:mm:ss Z" );
                    if( oldestDate.diff( storyDate ) < 0 ) {
                        stories.push( jsonRSS.query.results.item[i].title );
                    }
                }

                news[ index ] = stories;

                // Update the "last updated" information with a count of all stories from all feeds
                var newsCountTotal = 0;
                for( var i=0; i < news.length; i++ ) {
                    newsCountTotal += news[i].length;
                }

                $('.luRSS').updateWithText('rss (' + newsCountTotal + ' articles/' + news.length + ' feeds): ' + moment().format('h:mm a ddd MMM D YYYY'), 1000);
            }
        });
    }

    (function showNews() {
        var initialFeed = newsFeedIndex;

        if( news.length == 0 ) {
            // No news; nothing to do
            return;
        }

        // Find the next story
        for( var i=0; i < news.length+1; i++ ) {
            var newsFeed = news[ newsFeedIndex ];

            // Fix undefined entries
            if( newsFeed === undefined )
                continue;

            // Skip empty feeds
            if( newsFeed.length == 0 ) {
                if( ++newsFeedIndex == news.length )
                    newsFeedIndex = 0;

                newsStoryIndex = 0;
                continue;
            }

            // Check for the last story in the feed
            if( newsStoryIndex >= newsFeed.length  ) {        // newsStoryIndex can become greater than the array if the story count updated behind the scenes
                newsStoryIndex = 0;

                if( ++newsFeedIndex >= news.length ) {
                    newsFeedIndex = 0;
                    continue;
                }
            }
        }

        if( news[ newsFeedIndex ].length == 0 ) {
            // No stories
            setTimeout( showNews, 1000 );
            return;
        }

        // Get the title text
        var i = 0;
        for( var key in feedURLs ) {
            if( i == newsFeedIndex )
                break;

            i++;
        }

        $('.newsTitle').updateWithTextForce(key + '<hr width="20%" style="opacity:0.3">', 2000, true);

        // Draw feed/story dots
        var newsDots = ""
        for( i=0; i < news.length; i++ ) {
            if( i == newsFeedIndex )
                newsDots += '<span class="dimmed">&bull;</span>'
            else
                newsDots += '<span class="xxdimmed">&bull;</span>'
        }
        newsDots += "  ";

        for( i=0; i < news[ newsFeedIndex ].length; i++ ) {
            if( i == newsStoryIndex )
                newsDots += '<span class="dimmed">&bull;</span>'
            else
                newsDots += '<span class="xxdimmed">&bull;</span>'
        }

        $('.newsDots').updateWithTextForce(newsDots, 2000, true);

        // Get the story text
        var newsFeed = news[ newsFeedIndex ];
        newsStory = newsFeed[ newsStoryIndex ];

        var nextTimeout = 1000;                            // Used in error cases where newsStory is undefined for some reason
        if( typeof newsStory != 'undefined') {
            $('.news').updateWithText( newsStory, 2000 );

            nextTimeout = 5500 + (newsStory.length * 20);
        }

        // Set up for the next story
        newsStoryIndex++;
        setTimeout( showNews, nextTimeout  );            // Length of the headline modifies how long it stays on screen
    })();

});
TIA
My head hurts, I'm cross eyed and rather frustrated at my lack of knowledge!
 

Johnatan56

Honorary Master
Joined
Aug 23, 2013
Messages
22,033
#5
Dumb question, but why don't you just use e.g. accuweather or weather.com's free tier API?
E.g. current weather: https://samples.openweathermap.org/...don,uk&appid=b6907d289e10d714a6e88b30761fae22

Then use the JSON.parse() functions into an array and you're good to go.
https://www.w3schools.com/js/js_json_parse.asp

as a guide

And then probably use Google's news API.

Not really solving your problem, just saying you can use other sources besides Yahoo.
Documentation is pretty good: https://newsapi.org/docs
 

Düber

Well-Known Member
Joined
May 17, 2018
Messages
144
#6
Thanks for that, going to look at it now.
The weather is not a problem as it uses an API from Dark sky and the guy has it nicely set out in a beautiful graph with highs and lows, expected rain, week ahead etc. so there I'm sorted and happy as it is very dynamic.

My biggest stumbling block is JSON and not understanding it. there is obviously something fundamental that I'm not getting.
I have made a PHP that teases the headlines and published dates from the various sources, and can echo it out in html or JSON format but that is where I'm currently stuck.
I wish I had known they(Yahoo) where going to stop it as I could of perhaps seen what the reply looked like and tried to emulate it.
But let me look at Google's API perhaps it will be easier for me.

Edit- Thanks for the links, they look like what I need. Might get news back the notice board this weekend!:)
 
Last edited:

Johnatan56

Honorary Master
Joined
Aug 23, 2013
Messages
22,033
#7
Thanks for that, going to look at it now.
The weather is not a problem as it uses an API from Dark sky and the guy has it nicely set out in a beautiful graph with highs and lows, expected rain, week ahead etc. so there I'm sorted and happy as it is very dynamic.

My biggest stumbling block is JSON and not understanding it. there is obviously something fundamental that I'm not getting.
I have made a PHP that teases the headlines and published dates from the various sources, and can echo it out in html or JSON format but that is where I'm currently stuck.
I wish I had known they(Yahoo) where going to stop it as I could of perhaps seen what the reply looked like and tried to emulate it.
But let me look at Google's API perhaps it will be easier for me.

Edit- Thanks for the links, they look like what I need. Might get news back the notice board this weekend!:)
Well, think of JSON as just being an array with key:value pairs in it. That value might be more arrays, or not. The only way you'll know what is in it is by checking the documentation.

In PHP you can make it an associative array when it decodes it, https://stackoverflow.com/questions/29308898/how-do-i-extract-data-from-json-with-php , which will make it easier.
E.g. from my above link,
1547239906935.png
is what the array looks like, if you then did a $parsedData = json_decode($data, true); then
<?= $parsedData['name'] ?>
would echo London. Above image is from opening it in FF. You can check such requests made in JS/client side as well using the networking tab, find the page it gets it from there.

EDIT: Oh, and multi dimensional array, e.g. to access what country would be something like $parsedData['sys']['country'] and the result would be GB. Really not that complex once you get your head around the structure, actually way easier than being forced to loop forever searching through stuff. Should have seen the horror code of some first years I reviewed who decided that it was a good idea to throw everything in a text file with a tokenizer and then get stuff according to the n position /shudder.
 
Last edited:

Düber

Well-Known Member
Joined
May 17, 2018
Messages
144
#8
Well, think of JSON......
Thank you very much for the explanation and the link, I have obviously been searching for and trying things in the wrong way(ask the right question:)...).
I think I can see now what I'm doing wrong, I'm going to give it a good go over the weekend.
The Google API looks like a very good option too, and like you said well documented with example code.

:ROFL:The world has moved on since BASIC.:ROFL:
 

Düber

Well-Known Member
Joined
May 17, 2018
Messages
144
#9
Screenshot from 2019-01-13 21-41-26.jpg


Thank you @Johnatan56 got it working again, used the google api in the end.
Learned a serious amount about Java,PHP and json.

TL-DR - :) If someone(foolishly) wants to hire me as a dev, make sure you only pay on delivered results and not on hours or lines. :)

Ps. @ekske1 you may leave the saferoom now!:)
 
Top