Web Scraping
Contents
Web Scraping#
Jupyter Info
Reminder, that on this site the Jupyter Notebooks are read-only and you can’t interact with them. Click the button above to launch an interactive version of this notebook.
With Binder, you get a temporary Jupyter Notebook website that opens with this notebook. Any code you write will be lost when you close the tab. Make sure to download the notebook so you can save it for later!
With Colab, it will open Google Colaboratory. You can save the notebook there to your Google Drive. If you don’t save to your Drive, any code you write will be lost when you close the tab. You can find the data files for this notebook below:
You will need to run all the cells of the notebook to see the output. You can do this with hitting Shift-Enter on each cell or clicking the “Run All” button above.
APIs#
The first web technology we talked about are the Application Programming Interface (API). Any API is generally a URL you can go to that returns data (as opposed to a web-page like facebook.com). We looked at an API that returns the position of the International Space Station. This API is located at http://api.open-notify.org/iss-now.json and returns data in the JSON format. JSON is basically just a python lists and dictionaries with keys and values.
requests is a very popular Python library that lets you fetch data from a URL.
import requests
response = requests.get('http://api.open-notify.org/iss-now.json')
This returns a “response” object that has information about the response like it’s status code and data
response.status_code
200
You can view the data with the content attribute, but this returns a string which is not very helpful. Instead, we can use the json method to convert the string to a python dictionary.
response.content
b'{"message": "success", "timestamp": 1591032742, "iss_position": {"longitude": "-101.1647", "latitude": "-15.1609"}}'
d = response.json()
print(d)
print(d['timestamp'])
{'message': 'success', 'timestamp': 1591032742, 'iss_position': {'longitude': '-101.1647', 'latitude': '-15.1609'}}
1591032742
How did we know there was a key called 'timestamp'? This is part of the documentation of the API that can be found here.
To get more up-to-date data, we would have to make the request again. The code below makes 20 calls to the API and prints the latitude and longitude of the ISS.
for i in range(20):
response = requests.get('http://api.open-notify.org/iss-now.json')
if response.status_code == 200:
print(response.json()['iss_position'])
else:
print('Error')
{'longitude': '-101.1452', 'latitude': '-15.1361'}
{'longitude': '-101.1452', 'latitude': '-15.1361'}
{'longitude': '-101.1257', 'latitude': '-15.1113'}
{'longitude': '-101.1063', 'latitude': '-15.0864'}
{'longitude': '-101.1063', 'latitude': '-15.0864'}
{'longitude': '-101.0868', 'latitude': '-15.0616'}
{'longitude': '-101.0868', 'latitude': '-15.0616'}
{'longitude': '-101.0674', 'latitude': '-15.0369'}
{'longitude': '-101.0674', 'latitude': '-15.0369'}
{'longitude': '-101.0480', 'latitude': '-15.0120'}
{'longitude': '-101.0286', 'latitude': '-14.9872'}
{'longitude': '-101.0286', 'latitude': '-14.9872'}
{'longitude': '-101.0092', 'latitude': '-14.9624'}
{'longitude': '-101.0092', 'latitude': '-14.9624'}
{'longitude': '-100.9898', 'latitude': '-14.9376'}
{'longitude': '-100.9898', 'latitude': '-14.9376'}
{'longitude': '-100.9704', 'latitude': '-14.9128'}
{'longitude': '-100.9510', 'latitude': '-14.8880'}
{'longitude': '-100.9510', 'latitude': '-14.8880'}
{'longitude': '-100.9315', 'latitude': '-14.8632'}
Web Scraping#
APIs are fantastic because the return useful data that is generally well formatted! One problem is the API needs to be written by someone and they don’t always exist for the things you want. If there is no API to access the data nicely, another approach is to “scrape” the data of a webpage.
In the rest of this example, we will try to scrape this webpage with the weather forecast so we can gather data about the weather for the rest of the week. This example is a bit lacking since we don’t show what you would do with this data, just how to get it. To know what you can do with the data, refer to the rest of the course where we learned how to process data we had; web-scraping is just a tool to gather more data.
To understand how to scrape a page, you have to understand what a webpage looks like. Please refer to the slides to see what a webpage looks like.
page = requests.get('https://forecast.weather.gov/MapClick.php?lat=47.6036&lon=-122.3294')
print(page.content)
b'<!DOCTYPE html>\n<html class="no-js">\n <head>\n <!-- Meta -->\n <meta name="viewport" content="width=device-width">\n <link rel="schema.DC" href="http://purl.org/dc/elements/1.1/" /><title>National Weather Service</title><meta name="DC.title" content="National Weather Service" /><meta name="DC.description" content="NOAA National Weather Service National Weather Service" /><meta name="DC.creator" content="US Department of Commerce, NOAA, National Weather Service" /><meta name="DC.date.created" scheme="ISO8601" content="" /><meta name="DC.language" scheme="DCTERMS.RFC1766" content="EN-US" /><meta name="DC.keywords" content="weather, National Weather Service" /><meta name="DC.publisher" content="NOAA\'s National Weather Service" /><meta name="DC.contributor" content="National Weather Service" /><meta name="DC.rights" content="http://www.weather.gov/disclaimer.php" /><meta name="rating" content="General" /><meta name="robots" content="index,follow" />\n\n <!-- Icons -->\n <link rel="shortcut icon" href="./images/favicon.ico" type="image/x-icon" />\n\n <!-- CSS -->\n <link rel="stylesheet" href="css/bootstrap-3.2.0.min.css">\n <link rel="stylesheet" href="css/bootstrap-theme-3.2.0.min.css">\n <link rel="stylesheet" href="css/font-awesome-4.3.0.min.css">\n <link rel="stylesheet" href="css/ol-4.6.4.css" type="text/css">\n <link rel="stylesheet" type="text/css" href="css/mapclick.css" />\n <!--[if lte IE 7]><link rel="stylesheet" type="text/css" href="css/bootstrap-ie7.css" /><![endif]-->\n <!--[if lte IE 9]><link rel="stylesheet" type="text/css" href="css/mapclick-ie.css" /><![endif]-->\n <link rel="stylesheet" type="text/css" href="css/print.css" />\n <link rel="stylesheet" type="text/css" href="css/search.css" />\n\n <!-- Javascript -->\n <script type="text/javascript" src="js/lib/modernizr-2.8.3.js"></script>\n <script type="text/javascript" src="js/lib/json3-3.3.2.min.js"></script>\n <script type="text/javascript" src="js/lib/jquery-1.11.3.min.js"></script>\n <script type="text/javascript" src="js/lib/jquery.hoverIntent-1.8.1.min.js"></script>\n <script type="text/javascript" src="js/lib/bootstrap-3.2.0.min.js"></script>\n <script type="text/javascript" src="js/lib/ol-4.6.4.js"></script>\n <!--[if lte IE 8]><script type="text/javascript" src="js/respond.min.js"></script><![endif]-->\n <script type="text/javascript" src="js/jquery.autocomplete.min.js"></script>\n <script type="text/javascript" src="js/cfisurvey/cfi.js?v2"></script>\n <script type="text/javascript" src="js/forecast.esri.js"></script>\n <script type="text/javascript" src="js/forecast.search.js"></script>\n <script type="text/javascript" src="js/forecast.openlayers.js"></script>\n <script type="text/javascript" src="js/browserSniffer.js"></script>\n <script type="text/javascript" src="js/federated-analytics.js"></script>\n <script type="javascript">\n// ForeSee Staging Embed Script v2.01\n// DO NOT MODIFY BELOW THIS LINE *****************************************\n;(function (g) {\n var d = document, am = d.createElement(\'script\'), h = d.head || d.getElementsByTagName("head")[0], fsr = \'fsReady\',\n aex = { \n "src": "//gateway.foresee.com/sites/weather-gov/production/gateway.min.js",\n "type": "text/javascript", \n "async": "true", \n "data-vendor": "fs", \n "data-role": "gateway" \n };\n for (var attr in aex) { am.setAttribute(attr, aex[attr]); } h.appendChild(am); g[fsr] || (g[fsr] = function () { var aT = \'__\' + fsr + \'_stk__\'; g[aT] = g[aT] || []; g[aT].push(arguments); });\n})(window);\n// DO NOT MODIFY ABOVE THIS LINE *****************************************\n</script>\n <script type="text/javascript">\n (function (i, s, o, g, r, a, m) {\n i[\'GoogleAnalyticsObject\'] = r;\n i[r] = i[r] || function () {\n (i[r].q = i[r].q || []).push(arguments)\n }, i[r].l = 1 * new Date();\n a = s.createElement(o),\n m = s.getElementsByTagName(o)[0];\n a.async = 1;\n a.src = g;\n m.parentNode.insertBefore(a, m)\n })(window, document, \'script\', \'//www.google-analytics.com/analytics.js\', \'ga\');\n\n ga(\'create\', \'UA-40768555-1\', \'weather.gov\', {\'sampleRate\': 6});\n ga(\'set\', \'anonymizeIp\', true);\n ga(\'require\', \'linkid\');\n ga(\'send\', \'pageview\');\n </script>\n\n </head>\n <body>\n <main class="container">\n \t\t<header class="row clearfix" id="page-header">\r\n\t\t\t<a href="http://www.noaa.gov" id="header-noaa" class="pull-left"><img src="/css/images/header_noaa.png" alt="National Oceanic and Atmospheric Administration"/></a>\r\n\t\t\t<a href="http://www.weather.gov" id="header-nws" class="pull-left"><img src="/css/images/header_nws.png" alt="National Weather Service"/></a>\r\n\t\t\t<a href="http://www.commerce.gov" id="header-doc" class="pull-right"><img src="/css/images/header_doc.png" alt="United States Department of Commerce"/></a>\r\n\t\t</header>\r\n\t\t\n <nav class="navbar navbar-default row" role="navigation">\r\n <div class="container-fluid">\r\n <div class="navbar-header">\r\n <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#top-nav">\r\n <span class="sr-only">Toggle navigation</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="collapse navbar-collapse" id="top-nav">\r\n <ul class="nav navbar-nav">\r\n <li><a href="http://www.weather.gov">HOME</a></li>\r\n <li class="dropdown"><a href="http://www.weather.gov/forecastmaps" class="dropdown-toggle" data-toggle="dropdown">FORECAST <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="http://www.weather.gov">Local</a></li><li><a href="http://digital.weather.gov">Graphical</a></li><li><a href="http://www.aviationweather.gov/">Aviation</a></li><li><a href="http://www.nws.noaa.gov/om/marine/home.htm">Marine</a></li><li><a href="http://water.weather.gov/ahps/">Rivers and Lakes</a></li><li><a href="http://www.nhc.noaa.gov/">Hurricanes</a></li><li><a href="http://www.spc.noaa.gov/">Severe Weather</a></li><li><a href="http://www.weather.gov/fire/">Fire Weather</a></li><li><a href="https://www.esrl.noaa.gov/gmd/grad/solcalc/sunrise.html">Sun/Moon</a></li><li><a href="http://www.cpc.ncep.noaa.gov/">Long Range Forecasts</a></li><li><a href="http://www.cpc.ncep.noaa.gov">Climate Prediction</a></li><li><a href="https://www.swpc.noaa.gov/">Space Weather</a></li></ul> </li>\r\n <li class="dropdown"><a href="https://w2.weather.gov/climate" class="dropdown-toggle" data-toggle="dropdown">PAST WEATHER <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="https://w2.weather.gov/climate/">Past Weather</a></li><li><a href="https://w2.weather.gov/climate/">Heating/Cooling Days</a></li><li><a href="https://w2.weather.gov/climate/">Monthly Temperatures</a></li><li><a href="https://w2.weather.gov/climate/">Records</a></li><li><a href="http://aa.usno.navy.mil/">Astronomical Data</a></li></ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/safety" class="dropdown-toggle" data-toggle="dropdown">SAFETY <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="https://www.weather.gov/safety/flood">Floods</a></li><li><a href="https://www.weather.gov/safety/tsunami">Tsunamis</a></li><li><a href="https://www.weather.gov/safety/beachhazards">Beach Hazards</a></li><li><a href="https://www.weather.gov/safety/wildfire">Wildfire</a></li><li><a href="https://www.weather.gov/safety/cold">Cold</a></li><li><a href="https://www.weather.gov/safety/tornado">Tornadoes</a></li><li><a href="https://www.weather.gov/safety/fog">Fog</a></li><li><a href="https://www.weather.gov/safety/airquality">Air Quality</a></li><li><a href="https://www.weather.gov/safety/heat">Heat</a></li><li><a href="https://www.weather.gov/safety/hurricane">Hurricanes</a></li><li><a href="https://www.weather.gov/safety/lightning">Lightning</a></li><li><a href="https://www.weather.gov/safety/safeboating">Safe Boating</a></li><li><a href="https://www.weather.gov/safety/ripcurrent">Rip Currents</a></li><li><a href="https://www.weather.gov/safety/thunderstorm">Thunderstorms</a></li><li><a href="https://www.weather.gov/safety/space">Space Weather</a></li><li><a href="https://www.weather.gov/safety/heat-uv">Sun (Ultraviolet Radiation)</a></li><li><a href="http://www.weather.gov/safetycampaign">Safety Campaigns</a></li><li><a href="https://www.weather.gov/safety/wind">Wind</a></li><li><a href="https://www.weather.gov/safety/drought">Drought</a></li><li><a href="https://www.weather.gov/safety/winter">Winter Weather</a></li></ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/informationcenter" class="dropdown-toggle" data-toggle="dropdown">INFORMATION <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="http://www.weather.gov/Owlie\'s">Owlie\'s Kids Page</a></li><li><a href="http://www.weather.gov/wrn/wea">Wireless Emergency Alerts</a></li><li><a href="https://www.weather.gov/owlie/publication_brochures">Brochures</a></li><li><a href="http://www.weather.gov/wrn/">Weather-Ready Nation</a></li><li><a href="https://www.weather.gov/coop/">Cooperative Observers</a></li><li><a href="http://www.weather.gov/briefing/">Daily Briefing</a></li><li><a href="http://www.nws.noaa.gov/om/hazstats.shtml">Damage/Fatality/Injury Statistics</a></li><li><a href="http://mag.ncep.noaa.gov/">Forecast Models</a></li><li><a href="https://www.weather.gov/gis">GIS Data Portal</a></li><li><a href="https://www.weather.gov/nwr/">NOAA Weather Radio</a></li><li><a href="http://weather.gov/publications">Publications</a></li><li><a href="http://www.weather.gov/SKYWARN">SKYWARN Storm Spotters</a></li><li><a href="http://www.weather.gov/StormReady">StormReady</a></li><li><a href="https://www.weather.gov/TsunamiReady/">TsunamiReady</a></li><li><a href="https://www.weather.gov/notification/">Service Change Notices</a></li></ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/owlie" class="dropdown-toggle" data-toggle="dropdown">EDUCATION <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="https://www.weather.gov/wrn/force">Be A Force of Nature</a></li><li><a href="http://www.weather.gov/owlie">NWS Education Home</a></li></ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/contact-media/" class="dropdown-toggle" data-toggle="dropdown">NEWS <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="http://www.weather.gov/news">NWS News</a></li><li><a href="https://www.weather.gov/wrn/calendar">Events</a></li><li><a href="http://www.weather.gov/socialmedia">Social Media</a></li><li><a href="https://www.weather.gov/owlie/publication_brochures">Pubs/Brochures/Booklets </a></li><li><a href="http://www.noaa.gov/NOAA-Communications">NWS Media Contacts</a></li></ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/search" class="dropdown-toggle" data-toggle="dropdown">SEARCH <span class="caret"></span></a><ul class="dropdown-menu" role="menu"> <li><!-- Begin search code -->\r\n <div id="site-search">\r\n <form method="get" action="//search.usa.gov/search" style="margin-bottom: 0; margin-top: 0;">\r\n <input type="hidden" name="v:project" value="firstgov" /> \r\n <label for="query">Search For</label> \r\n <input type="text" name="query" id="query" size="12" /> \r\n <input type="submit" value="Go" />\r\n <p>\r\n <input type="radio" name="affiliate" checked="checked" value="nws.noaa.gov" id="nws" /> \r\n <label for="nws" class="search-scope">NWS</label> \r\n <input type="radio" name="affiliate" value="noaa.gov" id="noaa" /> \r\n <label for="noaa" class="search-scope">All NOAA</label>\r\n </p>\r\n </form>\r\n </div>\r\n </li>\r\n </ul> </li>\r\n <li class="dropdown"><a href="http://www.weather.gov/about" class="dropdown-toggle" data-toggle="dropdown">ABOUT <span class="caret"></span></a><ul class="dropdown-menu" role="menu"><li><a href="http://www.weather.gov/about">About NWS</a></li><li><a href="http://www.weather.gov/organization">Organization</a></li><li><a href="https://www.weather.gov/media/wrn/NWS_Weather-Ready-Nation_Strategic_Plan_2019-2022.pdf">Strategic Plan</a></li><li><a href="https://sites.google.com/a/noaa.gov/nws-insider/">For NWS Employees</a></li><li><a href="http://www.weather.gov/international/">International</a></li><li><a href="http://www.weather.gov/organization">National Centers</a></li><li><a href="http://www.weather.gov/careers/">Careers</a></li><li><a href="http://www.weather.gov/contact">Contact Us</a></li><li><a href="https://w1.weather.gov/glossary">Glossary</a></li></ul> </li>\r\n </ul>\r\n </div>\r\n </div>\r\n </nav>\r\n \n\t <div class="contentArea">\n\t\t\t<!-- Start Forecastsearch -->\n\t<div class="" id="fcst-search">\n\t <form name="getForecast" id="getForecast" class="form-inline" role="form" action="https://forecast.weather.gov/zipcity.php" method="get">\n\t\t<div id="getfcst-body">\n\t\t <input name="inputstring" type="text" class="form-control" id="inputstring" placeholder="" />\n\t\t <input name="btnSearch" id="btnSearch" class="btn btn-default" type="submit" value="Go" />\n\t\t <div id="txtHelp"><a href="javascript:void(window.open(\'http://weather.gov/ForecastSearchHelp.html\',\'locsearchhelp\',\'status=0,toolbar=0,location=0,menubar=0,directories=0,resizable=1,scrollbars=1,height=500,width=530\').focus());">View Location Examples</a></div>\n\t\t</div>\n\t\t<div id="txtError">\n\t\t <div id="errorNoResults" style="display:none;">Sorry, the location you searched for was not found. Please try another search.</div>\n\t\t <div id="errorMultipleResults" style="display:none">Multiple locations were found. Please select one of the following:</div>\n\t\t <div id="errorChoices" style="display:none"></div>\n\t\t <input id="btnCloseError" type="button" value="Close" style="display:none" />\n\t\t</div>\n\t\t<div id="getfcst-head">\n\t\t <p>Your local forecast office is</p>\n\t\t <h3 id="getfcst-headOffice"></h3>\n\t\t</div>\n\t </form>\n\t</div>\n\t<!-- end Forecastsearch -->\n \n\t\t<link rel="stylesheet" type="text/css" href="/css/topnews.css">\n<div id="news-items">\n <div id="topnews">\n <div class="icon"><img src="/images/news-important.jpg"></div>\n <div class="body">\n <h1 style="font-size: 11pt;">Critical Fire Weather Threats and Dangerous Heat Continue Across the West</h1>\n <p>\n Dangerously hot temperatures will continue early this week over the Desert Southwest, with critical fire weather threats from the same region northward into the Great Basin. Further north, severe thunderstorms will be possible across the northern Rockies and Plains today, expanding into the Upper Midwest on Tuesday.\n <a href="http://www.wpc.ncep.noaa.gov/discussions/hpcdiscussions.php?disc=pmdspd" target="_blank">Read More ></a>\n </p>\n </div>\n</div>\n\n</div>\n\t\t<script type="text/javascript">(function ($) { var topnews = $("#topnews"); topnews.hide(); $.get("siteNews.php", {a:"sew"},function(response){ if (response !== "false") topnews.replaceWith($(response)); topnews.show(); }); })(jQuery);</script><!-- PageFormat-Land -->\n<script language=javascript>document.title = $(\'<div/>\').html(\'7-Day Forecast for Latitude 47.61°N and Longitude 122.32°W (Elev. 240 ft)\').text();</script><img src="images/track_land_point.png" style="display:none;" />\n<div id="quickLinks">\n\t<span class="lang-spanish"><a href="//forecast.weather.gov/MapClick.php?lat=47.6036&lon=-122.3294&lg=sp">En Español</a></span>\n\t<div class="addthis_toolbox addthis_default_style addthis-forecast">\n\t <a href="//www.addthis.com/bookmark.php?v=250&pubid=ra-5127a6364d551d04" class="addthis_button_compact">Share</a>\n\t <span class="addthis_separator">|</span>\n\t <a class="addthis_button_preferred_1"></a>\n\t <a class="addthis_button_preferred_2"></a>\n\t <a class="addthis_button_preferred_3"></a>\n\t <a class="addthis_button_preferred_4"></a>\n\t <a class="addthis_button_preferred_5"></a>\n\t</div>\n\t<script type="text/javascript">\n\t\tvar addthis_config = addthis_config || {data_track_addressbar:true, pubid: \'xa-4b05b2d91f18c9cc\'};\n\t $(document).ready(function(){\n\t\t\tjQuery.ajax({\n\t\t\t\turl: "//s7.addthis.com/js/300/addthis_widget.js#async=1",\n\t\t\t\tdataType: "script",\n\t\t\t\tcache: false\n\t\t\t});\n\t });\n\t</script>\n</div>\n\n<!-- Current Conditions -->\n<div id="current-conditions" class="panel panel-default">\n\n\t<!-- Current Conditions header row -->\n <div class="panel-heading">\n\t\t<div>\n\t\t <b>Current conditions at</b>\n\t\t <h2 class="panel-title">UNIVERSITY OF WASHINGTON (UWASH)</h2>\n\t\t <span class="smallTxt"><b>Lat: </b>47.65°N<b>Lon: </b>122.31°W<b>Elev: </b>190ft.</span>\n\t </div>\n </div>\n <div class="panel-body" id="current-conditions-body">\n\t\t<!-- Graphic and temperatures -->\n\t\t<div id="current_conditions-summary" class="pull-left" >\n\t\t \t\t <p class="myforecast-current">NA</p>\n\t\t <p class="myforecast-current-lrg">59°F</p>\n\t\t <p class="myforecast-current-sm">15°C</p>\n\t\t</div>\n\t\t<div id="current_conditions_detail" class="pull-left">\n\t\t <table>\n <tr>\n <td class="text-right"><b>Humidity</b></td>\n <td>64%</td>\n </tr>\n <tr>\n <td class="text-right"><b>Wind Speed</b></td>\n <td>S 1 MPH</td>\n </tr>\n <tr>\n <td class="text-right"><b>Barometer</b></td>\n <td>1017.27 mb</td>\n </tr>\n <tr>\n <td class="text-right"><b>Dewpoint</b></td>\n <td>47°F (8°C)</td>\n </tr>\n <tr>\n <td class="text-right"><b>Visibility</b></td>\n <td>NA</td>\n </tr>\n <tr>\n <td class="text-right"><b>Last update</b></td>\n <td>\n 01 Jun 08:55 AM PDT </td>\n </tr>\n\t\t </table>\n\t\t</div>\n\t\t<div id="current_conditions_station">\n\t\t <div class="current-conditions-extra">\n <!-- Right hand section -->\n <p class="moreInfo"><b>More Information:</b></p><p><a id="localWFO" href="https://www.weather.gov/sew" title="Seattle, WA"><span class="hideText">Local</span> Forecast Office</a><a id="moreWx" href="https://www.weather.gov/wrh/LocalWeather?zone=WAZ558">More Local Wx</a><a href="https://www.wrh.noaa.gov/mesowest/getobext.php?wfo=sew&sid=UWASH&num=72&raw=0">3 Day History</a><a id="mobileWxLink" href="//mobile.weather.gov/index.php?lat=47.6036&lon=-122.3294&unit=0&lg=english">Mobile Weather</a><a id="wxGraph" href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=graphical">Hourly <span class="hideText">Weather </span>Forecast</a></p>\t\t </div>\n\t\t<!-- /current_conditions_station -->\n\t </div>\n\t <!-- /current-conditions-body -->\n\t</div>\n<!-- /Current Conditions -->\n</div>\n\n<!-- 7-Day Forecast -->\n<div id="seven-day-forecast" class="panel panel-default">\n <div class="panel-heading">\n\t<b>Extended Forecast for</b>\n\t<h2 class="panel-title">\n\t \t Downtown Seattle WA\t</h2>\n </div>\n <div class="panel-body" id="seven-day-forecast-body">\n\t\t\t<div id="seven-day-forecast-container"><ul id="seven-day-forecast-list" class="list-unstyled"><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Today<br><br></p>\n<p><img src="newimages/medium/few.png" alt="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " title="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " class="forecast-icon"></p><p class="short-desc">Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Tonight<br><br></p>\n<p><img src="newimages/medium/nfew.png" alt="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. " title="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. " class="forecast-icon"></p><p class="short-desc">Mostly Clear</p><p class="temp temp-low">Low: 50 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Tuesday<br><br></p>\n<p><img src="newimages/medium/bkn.png" alt="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. " title="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. " class="forecast-icon"></p><p class="short-desc">Partly Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Tuesday<br>Night</p>\n<p><img src="newimages/medium/nbkn.png" alt="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. " title="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. " class="forecast-icon"></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 53 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Wednesday<br><br></p>\n<p><img src="newimages/medium/bkn.png" alt="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. " title="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. " class="forecast-icon"></p><p class="short-desc">Partly Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Wednesday<br>Night</p>\n<p><img src="newimages/medium/nbkn.png" alt="Wednesday Night: Mostly cloudy, with a low around 51." title="Wednesday Night: Mostly cloudy, with a low around 51." class="forecast-icon"></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 51 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Thursday<br><br></p>\n<p><img src="newimages/medium/sct.png" alt="Thursday: Mostly sunny, with a high near 70." title="Thursday: Mostly sunny, with a high near 70." class="forecast-icon"></p><p class="short-desc">Mostly Sunny</p><p class="temp temp-high">High: 70 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Thursday<br>Night</p>\n<p><img src="newimages/medium/nbkn.png" alt="Thursday Night: Mostly cloudy, with a low around 52." title="Thursday Night: Mostly cloudy, with a low around 52." class="forecast-icon"></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 52 °F</p></div></li><li class="forecast-tombstone">\n<div class="tombstone-container">\n<p class="period-name">Friday<br><br></p>\n<p><img src="newimages/medium/shra.png" alt="Friday: A chance of showers after 11am. Partly sunny, with a high near 70." title="Friday: A chance of showers after 11am. Partly sunny, with a high near 70." class="forecast-icon"></p><p class="short-desc">Chance<br>Showers</p><p class="temp temp-high">High: 70 °F</p></div></li></ul></div>\n<script type="text/javascript">\n// equalize forecast heights\n$(function () {\n\tvar maxh = 0;\n\t$(".forecast-tombstone .short-desc").each(function () {\n\t\tvar h = $(this).height();\n\t\tif (h > maxh) { maxh = h; }\n\t});\n\t$(".forecast-tombstone .short-desc").height(maxh);\n});\n</script>\t</div>\n</div>\n\n<!-- Everything between 7-Day Forecast and Footer goes in this row -->\n<div id="floatingDivs" class="row">\n <!-- Everything on the left-hand side -->\n <div class="col-md-7 col-lg-8">\n <!-- Detailed Forecast -->\n <div id="detailed-forecast" class="panel panel-default">\n\t <div class="panel-heading">\n <h2 class="panel-title">Detailed Forecast</h2>\n </div>\n\t <div class="panel-body" id="detailed-forecast-body">\n <div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Today</b></div><div class="col-sm-10 forecast-text">Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. </div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Tonight</b></div><div class="col-sm-10 forecast-text">Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. </div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Tuesday</b></div><div class="col-sm-10 forecast-text">Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. </div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Tuesday Night</b></div><div class="col-sm-10 forecast-text">Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. </div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Wednesday</b></div><div class="col-sm-10 forecast-text">Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. </div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Wednesday Night</b></div><div class="col-sm-10 forecast-text">Mostly cloudy, with a low around 51.</div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Thursday</b></div><div class="col-sm-10 forecast-text">Mostly sunny, with a high near 70.</div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Thursday Night</b></div><div class="col-sm-10 forecast-text">Mostly cloudy, with a low around 52.</div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Friday</b></div><div class="col-sm-10 forecast-text">A chance of showers after 11am. Partly sunny, with a high near 70.</div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Friday Night</b></div><div class="col-sm-10 forecast-text">A chance of showers. Mostly cloudy, with a low around 52.</div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Saturday</b></div><div class="col-sm-10 forecast-text">Showers. Mostly cloudy, with a high near 65.</div></div><div class="row row-even row-forecast"><div class="col-sm-2 forecast-label"><b>Saturday Night</b></div><div class="col-sm-10 forecast-text">Showers likely. Mostly cloudy, with a low around 50.</div></div><div class="row row-odd row-forecast"><div class="col-sm-2 forecast-label"><b>Sunday</b></div><div class="col-sm-10 forecast-text">Showers. Mostly cloudy, with a high near 64.</div></div> </div>\n\t</div>\n\t<!-- /Detailed Forecast -->\n\n \n <!-- Additional Forecasts and Information -->\n <div id="additional_forecasts" class="panel panel-default">\n\t <div class="panel-heading">\n\t\t<h2 class="panel-title">Additional Forecasts and Information</h2>\n\t </div>\n\n\t <div class="panel-body" id="additional-forecasts-body">\n\t\t<p class="myforecast-location"><a href="MapClick.php?zoneid=WAZ558">Zone Area Forecast for Seattle and Vicinity, WA</a></p>\n <!-- First nine-ten links -->\n\t\t<div id="linkBlockContainer">\n\t\t <div class="linkBlock">\n <ul class="list-unstyled">\n <li><a href="//forecast.weather.gov/product.php?site=SEW&issuedby=SEW&product=AFD&format=CI&version=1&glossary=1">Forecast Discussion</a></li>\n <li><a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=text&TextType=2">Printable Forecast</a></li>\n <li><a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=text&TextType=1">Text Only Forecast</a></li>\n </ul>\n </div>\n\t\t <div class="linkBlock">\n <ul class="list-unstyled">\n <li><a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=graphical">Hourly Weather Forecast</a></li>\n <li><a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=digital">Tabular Forecast</a></li>\n <!-- <li><a href="afm/PointClick.php?lat=47.6036&lon=-122.3294">Quick Forecast</a></li> -->\n </ul>\n </div>\n\t\t <div class="linkBlock">\n <ul class="list-unstyled">\n <li><a href="//weather.gov/aq/probe_aq_data.php?latitude=47.6036&longitude=-122.3294">Air Quality Forecasts</a></li>\n <li><a href="MapClick.php?lat=47.6036&lon=-122.3294&FcstType=text&unit=1&lg=en">International System of Units</a></li>\n <li><a href="//www.wrh.noaa.gov/forecast/wxtables/index.php?lat=47.6036&lon=-122.3294">Forecast Weather Table Interface</a></li>\n </ul>\n\t\t </div>\n\t\t <!-- /First nine-ten links -->\n <!-- Additional links -->\n <div class="linkBlock"><ul class="list-unstyled"><li><a href="http://www.nws.noaa.gov/wtf/udaf/area/?site=sew" target="_self">User Defined Area</a></li></ul></div><div class="linkBlock"><ul class="list-unstyled"></ul></div><div class="linkBlock"><ul class="list-unstyled"></ul></div>\n\t\t</div> <!-- /linkBlockContainer -->\n\t </div><!-- /additional-forecasts-body-->\n\t</div> <!-- /additional_forecasts -->\n </div> <!-- /Everything on the left-hand side -->\n\n <!-- right-side-data -->\n <div class="col-md-5 col-lg-4" id="right-side-data">\n\t<div id="mapAndDescriptionArea">\n <!-- openlayer map -->\n <style>\n#custom-search{\ndisplay: block;\nposition: relative;\nz-index: 50;\ntop: 52px;\nleft: 60px;\n}\n#esri-geocoder-search{\ndisplay: block;\nposition: relative;\nz-index: 50;\ntop: 52px;\nleft: 60px;\n}\n#emap{\nmargin-top:15px;\ncursor:pointer;\nheight:370px;\nwidth:100%;\nborder: 1px solid #ccc;\nborder-radius: 3px;\n}\n#switch-basemap-container{\n}\n#basemap-selection-form ul{\nlist-style: none;\n margin: 0px;\n}\n#basemap-selection-form li{\nfloat: left;\n}\n.disclaimer{\nmargin-top:350px;\nmargin-left: 5px;\nz-index: 100;\nposition: absolute;\ntext-transform: none;\n}\n.esriAttributionLastItem{\ntext-transform: none;\n}\n.esriSimpleSlider div{\nheight:22px;\nline-height:20px;\nwidth:20px;\n}\n#point-forecast-map-label {\ntext-align:center;\nfont-weight:bold;\ncolor:black;\n}\n@media (max-width: 767px) {\n#emap{\nmargin-top:.5em;\nheight:270px;\n}\n.disclaimer{\nmargin-top:250px;\n}\n}\n</style>\n<!-- forecast-map -->\n<div class=\'point-forecast-map\'>\n <div class=\'point-forecast-map-header text-center\'>\n <div id="toolbar">\n \t<div id="switch-basemap-container">\n \t <div id="basemap-selection-form" title="Choose a Basemap">\n \t\t<div id="basemap-menu">\n \t\t <select name="basemap-selected" id="basemap-selected" autocomplete="off" title="Basemap Dropdown Menu">\n \t\t <option value="none">Select Basemap</option>\n \t\t <option value="topo" selected>Topographic</option>\n \t\t <option value="streets">Streets</option>\n \t\t <option value="satellite">Satellite</option>\n \t\t <option value="ocean">Ocean</option>\n \t\t </select>\n \t\t</div>\n \t </div>\n \t <div id="point-forecast-map-label">\n Click Map For Forecast\n </div>\n \t</div><!-- //#switch-basemap-container -->\n \t<div style="clear:both;"></div>\n </div><!-- //#toolbar -->\n </div><!-- //.point-forecast-map-header -->\n\n <div id="emap">\n <noscript><center><br><br><b>Map function requires Javascript and a compatible browser.</b></center></noscript>\n <div class="disclaimer"><a href=\'http://www.weather.gov/disclaimer#esri\'>Disclaimer</a></div>\n </div><!-- //#emap -->\n\n <div class="point-forecast-map-footer">\n <img src="./images/wtf/maplegend_forecast-area.gif" width="100" height="16" alt="Map Legend">\n </div><!-- //.point-forecast-map-footer -->\n\n</div> <!-- //.point-forecast-map -->\n<!-- //forecast-map -->\n <!-- //openlayer map -->\n\n\t <!-- About this Forecast -->\n <div id="about_forecast">\n <div class="fullRow">\n <div class="left">Point Forecast:</div>\n <div class="right">Downtown Seattle WA<br> 47.61°N 122.32°W (Elev. 240 ft)</div>\n </div>\n <div class="fullRow">\n <div class="left"><a target="_blank" href="//www.weather.gov/glossary/index.php?word=Last+update">Last Update</a>: </div>\n <div class="right">3:09 am PDT Jun 1, 2020</div>\n </div>\n <div class="fullRow">\n <div class="left"><a target="_blank" href="//www.weather.gov/glossary/index.php?word=forecast+valid+for">Forecast Valid</a>: </div>\n <div class="right">10am PDT Jun 1, 2020-6pm PDT Jun 7, 2020</div>\n </div>\n <div class="fullRow">\n <div class="left"> </div>\n <div class="right"><a href="https://forecast.weather.gov/product.php?site=SEW&issuedby=SEW&product=AFD&format=CI&version=1&glossary=1">Forecast Discussion</a></div>\n </div>\n <div class="fullRow">\n <div class="left"> </div>\n <div class="right">\n <a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=kml"><img src="/images/wtf/kml_badge.png" width="45" height="17" alt="Get as KML" /></a>\n <a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=dwml"><img src="/images/wtf/xml_badge.png" width="45" height="17" alt="Get as XML" /></a>\n </div>\n </div>\n </div>\n\t <!-- /About this Forecast -->\n\t</div>\n \n <!--additionalForecast-->\n <div class="panel panel-default" id="additionalForecast">\n <div class="panel-heading">\n <h2 class="panel-title">Additional Resources</h2>\n </div>\n <div class="panel-body">\n\n <!-- Radar & Satellite Images -->\n <div id="radar" class="subItem">\n <h4>Radar & Satellite Image</h4>\n <a href="//radar.weather.gov/radar.php?rid=atx&product=N0R&overlay=11101111&loop=no"><img src="//radar.weather.gov/Thumbs/ATX_Thumb.gif" class="radar-thumb" alt="Link to Local Radar Data" title="Link to Local Radar Data"></a> <a href="https://www.star.nesdis.noaa.gov/GOES/sector.php?sat=G17§or=pnw"><img src="https://cdn.star.nesdis.noaa.gov/GOES17/ABI/SECTOR/pnw/GEOCOLOR/600x600.jpg" class="satellite-thumb" alt="Link to Satellite Data" title="Link to Satellite Data"></a> </div>\n <!-- /Radar & Satellite Images -->\n <!-- Hourly Weather Forecast -->\n <div id="feature" class="subItem">\n <h4>Hourly Weather Forecast</h4>\n <a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=graphical"><img src="newimages/medium/hourlyweather.png" class="img-responsive" /></a>\n </div>\n <!-- /Hourly Weather Forecast -->\n <!-- NDFD -->\n <div id="NDFD" class="subItem">\n <h4>National Digital Forecast Database</h4>\n <div class="one-sixth-first"><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=MaxT"><img src="//graphical.weather.gov/images/thumbnail/latest_MaxMinT_pacnorthwest_thumbnail.png" border="0" alt="National Digital Forecast Database Maximum Temperature Forecast" title="National Digital Forecast Database Maximum Temperature Forecast" width="147" height="150"></a>\n\t \t\t\t<p><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=MaxT">High Temperature</a></p></div><div class="one-sixth-first"><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=Wx"><img src="//graphical.weather.gov/images/thumbnail/latest_Wx_pacnorthwest_thumbnail.png" border="0" alt="National Digital Forecast Database Weather Element Forecast" title="National Digital Forecast Database Weather Element Forecast" width="147" height="150"></a>\n\t \t\t\t<p><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=Wx">Chance of Precipitation</a></p></div> </div>\n <!-- /NDFD -->\n </div>\n </div>\n <!-- /additionalForecast -->\n\n </div>\n <!-- /col-md-4 -->\n <!-- /right-side-data -->\n <script language=\'javascript\'>$( document ).ready(function() { load_openlayers_map(\'\', \'\', \'\', \'{"centroid_lat":"47.6036","centroid_lon":"-122.3294","lat1":"47.5955","lon1":"-122.332","lat2":"47.616","lon2":"-122.338","lat3":"47.62","lon3":"-122.3075","lat4":"47.5995","lon4":"-122.3015"}\') });</script></div>\n<!-- /row -->\n\n\n</div>\n<!-- /PageFormat-Land -->\n\n\t </div>\n <footer>\n <div id="sitemap" class="sitemap-content row">\r\n <div class="col-xs-12">\r\n <div class="sitemap-columns">\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href=" http://www.nws.noaa.gov/climate/">PAST WEATHER</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-1">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-1">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://www.cpc.ncep.noaa.gov/products/MD_index.shtml">Climate Monitoring </a></li>\r\n <li><a href="https://w2.weather.gov/climate/">Past Weather </a></li>\r\n <li><a href=" http://www.nws.noaa.gov/climate/">Monthly Temps </a></li>\r\n <li><a href=" http://www.nws.noaa.gov/climate/">Records </a></li>\r\n <li><a href="https://www.esrl.noaa.gov/gmd/grad/solcalc/sunrise.html">Astronomical Data </a></li>\r\n <li><a href="https://www.climate.gov/maps-data/dataset/past-weather-zip-code-data-table">Certified Weather Data </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://alerts.weather.gov">ACTIVE ALERTS</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-2">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-2">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://alerts.weather.gov">Warnings By State</a></li>\r\n <li><a href=" http://www.wpc.ncep.noaa.gov/ww.shtml">Excessive Rainfall and Winter Weather Forecasts</a></li>\r\n <li><a href="http://water.weather.gov/ahps/?current_color=flood&current_type=all&fcst_type=obs&conus_map=d_map">River Flooding </a></li>\r\n <li><a href=" http://www.weather.gov">Latest Warnings</a></li>\r\n <li><a href=" http://www.spc.noaa.gov/products/outlook/">Thunderstorm/Tornado Outlook </a></li>\r\n <li><a href=" http://www.nhc.noaa.gov/">Hurricanes </a></li>\r\n <li><a href=" http://www.spc.noaa.gov/products/fire_wx/">Fire Weather Outlooks </a></li>\r\n <li><a href=" http://www.cpc.ncep.noaa.gov/products/stratosphere/uv_index/uv_alert.shtml">UV Alerts </a></li>\r\n <li><a href=" http://www.drought.gov/">Drought </a></li>\r\n <li><a href="http://www.swpc.noaa.gov/products/alerts-watches-and-warnings">Space Weather </a></li>\r\n <li><a href=" http://www.nws.noaa.gov/nwr/">NOAA Weather Radio </a></li>\r\n <li><a href=" http://alerts.weather.gov/">NWS CAP Feeds </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://www.weather.gov/current">CURRENT CONDITIONS</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-3">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-3">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://www.weather.gov/Radar">Radar </a></li>\r\n <li><a href="http://www.cpc.ncep.noaa.gov/products/monitoring_and_data/">Climate Monitoring </a></li>\r\n <li><a href=" http://water.weather.gov/ahps/">River Levels </a></li>\r\n <li><a href=" http://water.weather.gov/precip/">Observed Precipitation </a></li>\r\n <li><a href="https://www.wpc.ncep.noaa.gov/sfc/sfcobs/sfcobs.shtml">Surface Weather </a></li>\r\n <li><a href="http://www.spc.noaa.gov/obswx/maps/">Upper Air </a></li>\r\n <li><a href=" http://www.ndbc.noaa.gov/">Marine and Buoy Reports </a></li>\r\n <li><a href="http://www.nohrsc.noaa.gov/interactive/html/map.html">Snow Cover </a></li>\r\n <li><a href=" http://www.weather.gov/satellite">Satellite </a></li>\r\n <li><a href=" http://www.swpc.noaa.gov/">Space Weather </a></li>\r\n <li><a href="http://www.weather.gov/pr">International Observations</a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://weather.gov/forecastmaps">FORECAST</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-4">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-4">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://www.weather.gov/">Local Forecast </a></li>\r\n <li><a href="http://www.weather.gov/pr">International Forecasts</a></li>\r\n <li><a href=" http://www.spc.noaa.gov/">Severe Weather </a></li>\r\n <li><a href=" http://www.wpc.ncep.noaa.gov/">Current Outlook Maps </a></li>\r\n <li><a href="http://www.cpc.ncep.noaa.gov/products/Drought">Drought </a></li>\r\n <li><a href="http://www.weather.gov/fire">Fire Weather </a></li>\r\n <li><a href=" http://www.wpc.ncep.noaa.gov/">Fronts/Precipitation Maps </a></li>\r\n <li><a href=" http://www.nws.noaa.gov/forecasts/graphical/">Current Graphical Forecast Maps </a></li>\r\n <li><a href="http://water.weather.gov/ahps/forecasts.php">Rivers </a></li>\r\n <li><a href="https://www.weather.gov/marine/">Marine </a></li>\r\n <li><a href="https://ocean.weather.gov/marine_areas.php">Offshore and High Seas</a></li>\r\n <li><a href=" http://www.nhc.noaa.gov/">Hurricanes </a></li>\r\n <li><a href=" http://aviationweather.gov">Aviation Weather </a></li>\r\n <li><a href="http://www.cpc.ncep.noaa.gov/products/OUTLOOKS_index.shtml">Climatic Outlook </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://www.weather.gov/informationcenter">INFORMATION CENTER</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-5">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-5">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://www.spaceweather.gov">Space Weather </a></li>\r\n <li><a href="http://www.weather.gov/briefing/">Daily Briefing </a></li>\r\n <li><a href=" http://www.nws.noaa.gov/om/marine/home.htm">Marine </a></li>\r\n <li><a href="http://www.nws.noaa.gov/climate">Climate </a></li>\r\n <li><a href="http://www.weather.gov/fire">Fire Weather </a></li>\r\n <li><a href=" http://www.aviationweather.gov/">Aviation </a></li>\r\n <li><a href="http://mag.ncep.noaa.gov/">Forecast Models </a></li>\r\n <li><a href="http://water.weather.gov/ahps/">Water </a></li>\r\n <li><a href="https://www.weather.gov/gis/">GIS</a></li>\r\n <li><a href=" http://www.nws.noaa.gov/om/coop/">Cooperative Observers </a></li>\r\n <li><a href="https://www.weather.gov/skywarn/">Storm Spotters </a></li>\r\n <li><a href="http://www.tsunami.gov">Tsunami Warning System</a></li>\r\n <li><a href="http://water.noaa.gov/">National Water Center</a></li>\r\n <li><a href="http://www.weather.gov/pr">International Weather</a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://weather.gov/safety">WEATHER SAFETY</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-6">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-6">\r\n <ul class="list-unstyled">\r\n <li><a href="http://www.weather.gov/nwr/">NOAA Weather Radio</a></li>\r\n <li><a href="http://www.weather.gov/stormready/">StormReady</a></li>\r\n <li><a href="http://www.nws.noaa.gov/om/heat/index.shtml">Heat </a></li>\r\n <li><a href="https://www.weather.gov/safety/lightning">Lightning </a></li>\r\n <li><a href=" http://www.nhc.noaa.gov/prepare/">Hurricanes </a></li>\r\n <li><a href="http://www.nws.noaa.gov/om/thunderstorm/">Thunderstorms </a></li>\r\n <li><a href="https://www.weather.gov/safety/tornado">Tornadoes </a></li>\r\n <li><a href="https://www.weather.gov/safety/ripcurrent">Rip Currents </a></li>\r\n <li><a href="https://www.weather.gov/safety/flood">Floods </a></li>\r\n <li><a href="https://www.weather.gov/safety/tsunami">Tsunamis</a></li>\r\n <li><a href="https://www.weather.gov/tsunamiready/">TsunamiReady</a></li>\r\n <li><a href=" http://www.weather.gov/om/winter/index.shtml">Winter Weather </a></li>\r\n <li><a href="http://www.nws.noaa.gov/om/heat/uv.shtml">Ultra Violet Radiation </a></li>\r\n <li><a href=" http://www.weather.gov/airquality/">Air Quality </a></li>\r\n <li><a href=" http://www.weather.gov/om/hazstats.shtml">Damage/Fatality/Injury Statistics </a></li>\r\n <li><a href=" http://www.redcross.org/">Red Cross </a></li>\r\n <li><a href=" http://www.fema.gov/">Federal Emergency Management Agency (FEMA) </a></li>\r\n <li><a href=" http://www.weather.gov/om/brochures.shtml">Brochures </a></li>\r\n <li><a href="http://www.nws.noaa.gov/os/marine/safeboating/">Safe Boating</a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://weather.gov/news">NEWS</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-7">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-7">\r\n <ul class="list-unstyled">\r\n <li><a href=" http://weather.gov/news">Newsroom</a></li>\r\n <li><a href=" http://weather.gov/socialmedia">Social Media </a></li>\r\n <li><a href="http://www.nws.noaa.gov/com/weatherreadynation/calendar.html">Events</a></li>\r\n <li><a href=" http://www.weather.gov/om/brochures.shtml">Pubs/Brochures/Booklets </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://weather.gov/owlie">EDUCATION</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-8">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-8">\r\n <ul class="list-unstyled">\r\n <li><a href="http://weather.gov/owlie">NWS Education Home</a></li>\r\n <li><a href="http://www.nws.noaa.gov/com/weatherreadynation/force.html">Be A Force of Nature</a></li>\r\n <li><a href=" http://www.education.noaa.gov/Weather_and_Atmosphere/">NOAA Education Resources </a></li>\r\n <li><a href=" http://www.weather.gov/glossary/">Glossary </a></li>\r\n <li><a href="https://www.weather.gov/jetstream/">JetStream </a></li>\r\n <li><a href=" http://www.weather.gov/training/">NWS Training Portal </a></li>\r\n <li><a href="https://library.noaa.gov/">NOAA Library </a></li>\r\n <li><a href="http://weather.gov/owlie">For Students, Parents and Teachers</a></li>\r\n <li><a href="http://www.weather.gov/owlie/publication_brochures">Brochures </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n <div class="sitemap-section">\r\n <div class="panel-heading">\r\n <a class="sitemap-section-heading" href="http://weather.gov/about">ABOUT</a>\r\n <button type="button" class="menu-toggle pull-right" data-toggle="collapse" data-target="#sitemap-9">\r\n <span class="sr-only">Toggle menu</span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n <span class="icon-bar"></span>\r\n </button>\r\n </div>\r\n <div class="sitemap-section-body panel-body collapsable collapse" id="sitemap-9">\r\n <ul class="list-unstyled">\r\n <li><a href="http://weather.gov/organization">Organization </a></li>\r\n <li><a href="https://www.weather.gov/media/wrn/NWS_Weather-Ready-Nation_Strategic_Plan_2019-2022.pdf">Strategic Plan </a></li>\r\n <li><a href="https://sites.google.com/a/noaa.gov/nws-best-practices/">For NWS Employees </a></li>\r\n <li><a href="https://www.weather.gov/international/">International </a></li>\r\n <li><a href="http://www.ncep.noaa.gov/">National Centers </a></li>\r\n <li><a href=" http://www.weather.gov/tg/">Products and Services </a></li>\r\n <li><a href="http://www.weather.gov/careers/">Careers</a></li>\r\n <li><a href=" http://www.weather.gov/glossary/">Glossary </a></li>\r\n <li><a href="http://weather.gov/contact">Contact Us </a></li>\r\n </ul>\r\n </div>\r\n </div>\r\n </div>\r\n </div>\r\n </div>\r\n \n <!-- legal footer area -->\n \t\t<div class="footer-legal">\r\n\t\t\t<div id="footerLogo" class="col-xs-12 col-sm-2 col-md-2">\r\n\t\t\t\t<a href="http://www.usa.gov"><img src="/css/images/usa_gov.png" alt="usa.gov" width="110" height="30" /></a>\r\n\t\t\t</div>\r\n\t\t\t<div class="col-xs-12 col-sm-4 col-md-4">\r\n\t\t\t\t<ul class="list-unstyled footer-legal-content">\r\n\t\t\t\t<li><a href="http://www.commerce.gov">US Dept of Commerce</a></li>\r\n\t\t\t\t<li><a href="http://www.noaa.gov">National Oceanic and Atmospheric Administration</a></li>\r\n\t\t\t\t<li><a href="http://www.weather.gov">National Weather Service</a></li>\r\n\t\t\t\t<li><a href="https://www.weather.gov/sew">Seattle/Tacoma, WA</a></li><li><br /><a href="mailto:w-sew.webmaster@noaa.gov">Comments? Questions? Please Contact Us.</a></li>\t\t\t\t</ul>\r\n\t\t\t</div>\r\n\t\t\t<div class="col-xs-12 col-sm-3 col-md-3">\r\n\t\t\t\t<ul class="list-unstyled">\r\n\t\t\t\t\t<li><a href="https://www.weather.gov/disclaimer">Disclaimer</a></li>\r\n\t\t\t\t\t<li><a href="http://www.cio.noaa.gov/services_programs/info_quality.html">Information Quality</a></li>\r\n\t\t\t\t\t<li><a href="https://www.weather.gov/help">Help</a></li>\r\n\t\t\t\t\t<li><a href="http://www.weather.gov/glossary">Glossary</a></li>\r\n\t\t\t\t</ul>\r\n\t\t\t</div>\r\n\t\t\t<div class="col-xs-12 col-sm-3 col-md-3">\r\n\t\t\t\t<ul class="list-unstyled">\r\n\t\t\t\t\t<li><a href="https://www.weather.gov/privacy">Privacy Policy</a></li>\r\n\t\t\t\t\t<li><a href="https://www.noaa.gov/foia-freedom-of-information-act">Freedom of Information Act (FOIA)</a></li>\r\n\t\t\t\t\t<li><a href="https://www.weather.gov/about">About Us</a></li>\r\n\t\t\t\t\t<li><a href="https://www.weather.gov/careers">Career Opportunities</a></li>\r\n\t\t\t\t</ul>\r\n\t\t\t</div>\r\n\t\t</div>\r\n\t\t\n </footer>\n </main>\n </body>\n</html>\n'
This would be a pain to parse by hand, so instead we use a library that lets us look at the conents of the page. This library is called Beautiful Soup which can be used like below:
from bs4 import BeautifulSoup
soup = BeautifulSoup(page.content, 'html.parser')
To find the first paragraph in the page, we can write
soup.find('p')
<p>
<input checked="checked" id="nws" name="affiliate" type="radio" value="nws.noaa.gov"/>
<label class="search-scope" for="nws">NWS</label>
<input id="noaa" name="affiliate" type="radio" value="noaa.gov"/>
<label class="search-scope" for="noaa">All NOAA</label>
</p>
To find all the paragraphs, we would use find_all
soup.find_all('p')
[<p>
<input checked="checked" id="nws" name="affiliate" type="radio" value="nws.noaa.gov"/>
<label class="search-scope" for="nws">NWS</label>
<input id="noaa" name="affiliate" type="radio" value="noaa.gov"/>
<label class="search-scope" for="noaa">All NOAA</label>
</p>, <p>Your local forecast office is</p>, <p>
Dangerously hot temperatures will continue early this week over the Desert Southwest, with critical fire weather threats from the same region northward into the Great Basin. Further north, severe thunderstorms will be possible across the northern Rockies and Plains today, expanding into the Upper Midwest on Tuesday.
<a href="http://www.wpc.ncep.noaa.gov/discussions/hpcdiscussions.php?disc=pmdspd" target="_blank">Read More ></a>
</p>, <p class="myforecast-current">NA</p>, <p class="myforecast-current-lrg">59°F</p>, <p class="myforecast-current-sm">15°C</p>, <p class="moreInfo"><b>More Information:</b></p>, <p><a href="https://www.weather.gov/sew" id="localWFO" title="Seattle, WA"><span class="hideText">Local</span> Forecast Office</a><a href="https://www.weather.gov/wrh/LocalWeather?zone=WAZ558" id="moreWx">More Local Wx</a><a href="https://www.wrh.noaa.gov/mesowest/getobext.php?wfo=sew&sid=UWASH&num=72&raw=0">3 Day History</a><a href="//mobile.weather.gov/index.php?lat=47.6036&lon=-122.3294&unit=0&lg=english" id="mobileWxLink">Mobile Weather</a><a href="MapClick.php?lat=47.6036&lon=-122.3294&unit=0&lg=english&FcstType=graphical" id="wxGraph">Hourly <span class="hideText">Weather </span>Forecast</a></p>, <p class="period-name">Today<br/><br/></p>, <p><img alt="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " class="forecast-icon" src="newimages/medium/few.png" title="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. "/></p>, <p class="short-desc">Sunny</p>, <p class="temp temp-high">High: 66 °F</p>, <p class="period-name">Tonight<br/><br/></p>, <p><img alt="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. " class="forecast-icon" src="newimages/medium/nfew.png" title="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. "/></p>, <p class="short-desc">Mostly Clear</p>, <p class="temp temp-low">Low: 50 °F</p>, <p class="period-name">Tuesday<br/><br/></p>, <p><img alt="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. " class="forecast-icon" src="newimages/medium/bkn.png" title="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. "/></p>, <p class="short-desc">Partly Sunny</p>, <p class="temp temp-high">High: 66 °F</p>, <p class="period-name">Tuesday<br/>Night</p>, <p><img alt="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. " class="forecast-icon" src="newimages/medium/nbkn.png" title="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. "/></p>, <p class="short-desc">Mostly Cloudy</p>, <p class="temp temp-low">Low: 53 °F</p>, <p class="period-name">Wednesday<br/><br/></p>, <p><img alt="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. " class="forecast-icon" src="newimages/medium/bkn.png" title="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. "/></p>, <p class="short-desc">Partly Sunny</p>, <p class="temp temp-high">High: 66 °F</p>, <p class="period-name">Wednesday<br/>Night</p>, <p><img alt="Wednesday Night: Mostly cloudy, with a low around 51." class="forecast-icon" src="newimages/medium/nbkn.png" title="Wednesday Night: Mostly cloudy, with a low around 51."/></p>, <p class="short-desc">Mostly Cloudy</p>, <p class="temp temp-low">Low: 51 °F</p>, <p class="period-name">Thursday<br/><br/></p>, <p><img alt="Thursday: Mostly sunny, with a high near 70." class="forecast-icon" src="newimages/medium/sct.png" title="Thursday: Mostly sunny, with a high near 70."/></p>, <p class="short-desc">Mostly Sunny</p>, <p class="temp temp-high">High: 70 °F</p>, <p class="period-name">Thursday<br/>Night</p>, <p><img alt="Thursday Night: Mostly cloudy, with a low around 52." class="forecast-icon" src="newimages/medium/nbkn.png" title="Thursday Night: Mostly cloudy, with a low around 52."/></p>, <p class="short-desc">Mostly Cloudy</p>, <p class="temp temp-low">Low: 52 °F</p>, <p class="period-name">Friday<br/><br/></p>, <p><img alt="Friday: A chance of showers after 11am. Partly sunny, with a high near 70." class="forecast-icon" src="newimages/medium/shra.png" title="Friday: A chance of showers after 11am. Partly sunny, with a high near 70."/></p>, <p class="short-desc">Chance<br/>Showers</p>, <p class="temp temp-high">High: 70 °F</p>, <p class="myforecast-location"><a href="MapClick.php?zoneid=WAZ558">Zone Area Forecast for Seattle and Vicinity, WA</a></p>, <p><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=MaxT">High Temperature</a></p>, <p><a href="//graphical.weather.gov/sectors/pacnorthwest.php?element=Wx">Chance of Precipitation</a></p>]
We learned you can also specify an ID or a class to identify a tag in HTML. First, we select the element with the id “seven-day-forecast” and then try to find all of the items with the class “tombstone-container” inside that element.
seven_day = soup.find(id='seven-day-forecast')
seven_day
<div class="panel panel-default" id="seven-day-forecast">
<div class="panel-heading">
<b>Extended Forecast for</b>
<h2 class="panel-title">
Downtown Seattle WA </h2>
</div>
<div class="panel-body" id="seven-day-forecast-body">
<div id="seven-day-forecast-container"><ul class="list-unstyled" id="seven-day-forecast-list"><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Today<br/><br/></p>
<p><img alt="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " class="forecast-icon" src="newimages/medium/few.png" title="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. "/></p><p class="short-desc">Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Tonight<br/><br/></p>
<p><img alt="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. " class="forecast-icon" src="newimages/medium/nfew.png" title="Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. "/></p><p class="short-desc">Mostly Clear</p><p class="temp temp-low">Low: 50 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Tuesday<br/><br/></p>
<p><img alt="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. " class="forecast-icon" src="newimages/medium/bkn.png" title="Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. "/></p><p class="short-desc">Partly Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Tuesday<br/>Night</p>
<p><img alt="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. " class="forecast-icon" src="newimages/medium/nbkn.png" title="Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. "/></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 53 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Wednesday<br/><br/></p>
<p><img alt="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. " class="forecast-icon" src="newimages/medium/bkn.png" title="Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. "/></p><p class="short-desc">Partly Sunny</p><p class="temp temp-high">High: 66 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Wednesday<br/>Night</p>
<p><img alt="Wednesday Night: Mostly cloudy, with a low around 51." class="forecast-icon" src="newimages/medium/nbkn.png" title="Wednesday Night: Mostly cloudy, with a low around 51."/></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 51 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Thursday<br/><br/></p>
<p><img alt="Thursday: Mostly sunny, with a high near 70." class="forecast-icon" src="newimages/medium/sct.png" title="Thursday: Mostly sunny, with a high near 70."/></p><p class="short-desc">Mostly Sunny</p><p class="temp temp-high">High: 70 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Thursday<br/>Night</p>
<p><img alt="Thursday Night: Mostly cloudy, with a low around 52." class="forecast-icon" src="newimages/medium/nbkn.png" title="Thursday Night: Mostly cloudy, with a low around 52."/></p><p class="short-desc">Mostly Cloudy</p><p class="temp temp-low">Low: 52 °F</p></div></li><li class="forecast-tombstone">
<div class="tombstone-container">
<p class="period-name">Friday<br/><br/></p>
<p><img alt="Friday: A chance of showers after 11am. Partly sunny, with a high near 70." class="forecast-icon" src="newimages/medium/shra.png" title="Friday: A chance of showers after 11am. Partly sunny, with a high near 70."/></p><p class="short-desc">Chance<br/>Showers</p><p class="temp temp-high">High: 70 °F</p></div></li></ul></div>
<script type="text/javascript">
// equalize forecast heights
$(function () {
var maxh = 0;
$(".forecast-tombstone .short-desc").each(function () {
var h = $(this).height();
if (h > maxh) { maxh = h; }
});
$(".forecast-tombstone .short-desc").height(maxh);
});
</script> </div>
</div>
forecast_items = seven_day.find_all(class_='tombstone-container')
tonight = forecast_items[0]
tonight
<div class="tombstone-container">
<p class="period-name">Today<br/><br/></p>
<p><img alt="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " class="forecast-icon" src="newimages/medium/few.png" title="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. "/></p><p class="short-desc">Sunny</p><p class="temp temp-high">High: 66 °F</p></div>
To print it out a little nicer, we can use the prettify method
print(tonight.prettify())
<div class="tombstone-container">
<p class="period-name">
Today
<br/>
<br/>
</p>
<p>
<img alt="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. " class="forecast-icon" src="newimages/medium/few.png" title="Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. "/>
</p>
<p class="short-desc">
Sunny
</p>
<p class="temp temp-high">
High: 66 °F
</p>
</div>
Elements returned by find sometimes have attributes that you can inspect. For example, the img tags in the the forecast have a “title” attribute with information about the forecast.
tonight.find('img')['title']
'Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. '
Finding elements within an element can sometimes be tedious to do manually with find. BeautifulSoup also provides the select method that lets you find elements using a special syntax called “CSS Selectors”.
The follwoing line finds all the elements with the class “period-name” that are inside elements with the class “tombstone container”.
seven_day.select('.tombstone-container .period-name')
[<p class="period-name">Today<br/><br/></p>,
<p class="period-name">Tonight<br/><br/></p>,
<p class="period-name">Tuesday<br/><br/></p>,
<p class="period-name">Tuesday<br/>Night</p>,
<p class="period-name">Wednesday<br/><br/></p>,
<p class="period-name">Wednesday<br/>Night</p>,
<p class="period-name">Thursday<br/><br/></p>,
<p class="period-name">Thursday<br/>Night</p>,
<p class="period-name">Friday<br/><br/></p>]
To get the text inside each tag returned by select, we can use a loop to all the get_text method on each tag.
Below we get the following information for each time in the forecast
We get the name of the forecast time (called period)
We get the description of the forecast from the title property of the image inside the forecast
We get the forecast temperature
periods = [pt.get_text() for pt in seven_day.select('.tombstone-container .period-name')]
print(periods)
['Today', 'Tonight', 'Tuesday', 'TuesdayNight', 'Wednesday', 'WednesdayNight', 'Thursday', 'ThursdayNight', 'Friday']
titles = [img['title'] for img in seven_day.select('.tombstone-container img')]
print(titles)
['Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. ', 'Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. ', 'Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. ', 'Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. ', 'Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. ', 'Wednesday Night: Mostly cloudy, with a low around 51.', 'Thursday: Mostly sunny, with a high near 70.', 'Thursday Night: Mostly cloudy, with a low around 52.', 'Friday: A chance of showers after 11am. Partly sunny, with a high near 70.']
temps = [tt.get_text() for tt in seven_day.select('.tombstone-container .temp')]
print(temps)
['High: 66 °F', 'Low: 50 °F', 'High: 66 °F', 'Low: 53 °F', 'High: 66 °F', 'Low: 51 °F', 'High: 70 °F', 'Low: 52 °F', 'High: 70 °F']
All of the data together is shown below
print(titles)
print(temps)
print(periods)
['Today: Sunny, with a high near 66. Calm wind becoming west northwest around 6 mph in the afternoon. ', 'Tonight: Mostly clear, with a low around 50. North northwest wind around 7 mph becoming east northeast in the evening. ', 'Tuesday: Partly sunny, with a high near 66. East northeast wind around 5 mph becoming calm in the afternoon. ', 'Tuesday Night: Mostly cloudy, with a low around 53. North wind 5 to 7 mph becoming calm. ', 'Wednesday: Partly sunny, with a high near 66. Northeast wind 5 to 8 mph becoming northwest in the morning. ', 'Wednesday Night: Mostly cloudy, with a low around 51.', 'Thursday: Mostly sunny, with a high near 70.', 'Thursday Night: Mostly cloudy, with a low around 52.', 'Friday: A chance of showers after 11am. Partly sunny, with a high near 70.']
['High: 66 °F', 'Low: 50 °F', 'High: 66 °F', 'Low: 53 °F', 'High: 66 °F', 'Low: 51 °F', 'High: 70 °F', 'Low: 52 °F', 'High: 70 °F']
['Today', 'Tonight', 'Tuesday', 'TuesdayNight', 'Wednesday', 'WednesdayNight', 'Thursday', 'ThursdayNight', 'Friday']
The data is now stored as “parallel arrays”, where the value at index 0 in each array corresponds to one forecast. This would be a bit annoying to work with, so we put it in a pandas DataFrame
import pandas as pd
weather = pd.DataFrame({
'period': periods,
'temp': temps,
'desc': titles
})
weather
| period | temp | desc | |
|---|---|---|---|
| 0 | Today | High: 66 °F | Today: Sunny, with a high near 66. Calm wind b... |
| 1 | Tonight | Low: 50 °F | Tonight: Mostly clear, with a low around 50. N... |
| 2 | Tuesday | High: 66 °F | Tuesday: Partly sunny, with a high near 66. Ea... |
| 3 | TuesdayNight | Low: 53 °F | Tuesday Night: Mostly cloudy, with a low aroun... |
| 4 | Wednesday | High: 66 °F | Wednesday: Partly sunny, with a high near 66. ... |
| 5 | WednesdayNight | Low: 51 °F | Wednesday Night: Mostly cloudy, with a low aro... |
| 6 | Thursday | High: 70 °F | Thursday: Mostly sunny, with a high near 70. |
| 7 | ThursdayNight | Low: 52 °F | Thursday Night: Mostly cloudy, with a low arou... |
| 8 | Friday | High: 70 °F | Friday: A chance of showers after 11am. Partl... |