To help you learn about the big stories of the week, we’re kicking off a new series from CitizenTube, a YouTube channel focusing on global news and politics.
Everyday on the CitizenTube channel (and @CitizenTube on Twitter), along with our curation partners @storyful, we look at how the top news stories are covered on YouTube. Each week, starting today, we’ll post a weekly recap of the top news stories of the week, as seen through the lens of both citizen-reported footage and professional news coverage.
We followed the GOP candidates do battle in Florida as tensions grew on the US campaign trail, with Mitt Romney emerging as the comprehensive victor in the Sunshine state.
We watched social networking giant Facebook announce that it was going public, as it sought to raise $5bn in the largest IPO by a technology company.
We saw Novak Djokovic defeat Rafael Nadal to claim the Australian Open final, in a match that will go down in history as one of the greatest of all time.
And finally, we looked forward to Super Bowl XLVI, as some companies released advanced versions of their half-time commercials.
Here’s the latest installment of our monthly series on “search quality highlights,” with 17 new quality improvements to read about for January. In addition to this month’s big announcement of Search plus Your World, you’ll find short summaries of other changes to our high-quality sites algorithm, spelling systems, snippets, search preferences, speed, freshness and much more. It’s all part of our ongoing effort to be transparent about how search works and the ways Google is constantly evolving to answer your questions.
Here’s the list for January:
Fresher results. [launch codename “nftc”] We made several adjustments to the freshness algorithm that we released in November. These are minor updates to make sure we continue to give you the freshest, most relevant results.
Faster autocomplete. [launch codename “Snappy Suggest”, project codename “Suggest”] We made improvements to our autocomplete system to deliver your predicted queries much faster.
Autocomplete spelling corrections. [launch codename “Trivial”, project codename “Suggest”] This is an improvement to the spelling corrections used in autocomplete, making those corrections more consistent with the spelling corrections used in search. This launch targets corrections where the spelling change is very small.
Better spelling full-page replacement. [launch codenames “Oooni”, “sgap”, project codename “Full-Page Replacement”] When we’re confident in a spelling correction we automatically show results for the corrected query and let you know we’re “Showing results for [cheetah]” (rather than, say, “cheettah”). We made a couple of changes to improve the accuracy of this feature.
Better spelling corrections for rare queries. This change improves one of the models that we use to make spelling corrections. The result is more accurate spell corrections for a number of rare queries.
Improve detection of recurrent event pages. [launch codename “neseda”] We made several improvements to how we determine the date of a document. As a result, you’ll see fresher, more timely results, particularly for pages discussing recurring events.
High-quality sites algorithm improvements. [launch codenames “PPtl” and “Stitch”, project codename “Panda”] In 2011, we launched the Panda algorithm change, targeted at finding more high-quality sites. We improved how Panda interacts with our indexing and ranking systems, making it more integrated into our pipelines. We also released a minor update to refresh the data for Panda.
Cross-language refinements. [launch codename Xiangfan] Previously, we only generated related searches based on the display language. With this change, we also attempt to auto-detect the language of the original query to generate related search queries. Now, a user typing a query in French might see French query refinements, even if her language is set to English.
English on Google Saudi Arabia. Users in Saudi Arabia can now more easily choose an English interface to search on google.com.sa.
Improved scrolling for Image Search. Previously when you scrolled in Image Search, only the image results would move while the top and side menus were pinned in place. We changed the scrolling behavior to make it consistent with our main search results and the other search modes, where scrolling moves the entire page.
Improved image search quality. [launch codename “endearo”, project codename “Image Search”] This is a small improvement to our image search ranking algorithm. In particular, this change helps images with high-quality landing pages rank higher in our image search results.
More relevant related searches. Sometimes at the bottom of the screen you’ll see a section called “Searches related to” with other queries you may want to try. With this change, we’ve updated the model for generating related searches, resulting in more useful query refinements.
Blending of news results. [launch codename “final-destination”, project codename “Universal Search”] We improved our algorithm that decides which queries should show news results, making it more responsive to realtime trends. We also made an adjustment to how we blend news results in Universal Search. Both of these changes help news articles appear in your search results when they are relevant.
Automatically disable Google Instant based on computer speed. [project codename “Psychic Search”] Google Instant has long had the ability to automatically turn itself off if you’re on a slow internet connection. Now Instant can also turn itself off if your computer is slow. If Instant gets automatically disabled, we continue to check your computer speed and will re-enable Instant if your performance improves. We’ve also tweaked search preferences so you can always have Instant on or off, or have it change automatically.
And here’s a recap of some other January improvements we’ve already blogged about:
Stay tuned for more in the coming months. We have some exciting ideas for how we can continue innovating on transparency in search, and we hope you enjoy what’s on the way.
Computers store every piece of text using a “character encoding,” which gives a number to each character. For example, the byte 61 stands for ‘a’ and 62 stands for ‘b’ in the ASCII encoding, which was launched in 1963. Before the web, computer systems were siloed, and there were hundreds of different encodings. Depending on the encoding, C1 could mean any of ¡, Ё, Ą, Ħ, ‘, ”, or parts of thousands of characters, from æ to 品. If you brought a file from one computer to another, it could come out as gobbledygook.
Unicode was invented to solve that problem: to encode all human languages, from Chinese (中文) to Russian (русский) to Arabic (العربية), and even emoji symbols like or ; it encodes nearly 75,000 Chinese ideographs alone. In the ASCII encoding, there wasn’t even enough room for all the English punctuation (like curly quotes), while Unicode has room for over a million characters. Unicode was first published in 1991, coincidentally the year the World Wide Web debuted—little did anyone realize at the time they would be so important for each other. Today, people can easily share documents on the web, no matter what their language.
Every January, we look at the percentage of the webpages in our index that are in different encodings. Here’s what our data looks like with the latest figures*:
*Your mileage may vary: these figures may vary somewhat from what other search engines find. The graph lumps together encodings by script. We detect the encoding for each webpage; the ASCII pages just contain ASCII characters, for example. Thanks again to Erik van der Poel for collecting the data.
As you can see, Unicode has experienced an 800 percent increase in “market share” since 2006. Note that we separate out ASCII (~16 percent) since it is a subset of most other encodings. When you include ASCII, nearly 80 percent of web documents are in Unicode (UTF-8). The more documents that are in Unicode, the less likely you will see mangled characters (what Japanese call mojibake) when you’re surfing the web.
We’ve long used Unicode as the internal format for all the text Google searches and process: any other encoding is first converted to Unicode. Version 6.1 just released with over 110,000 characters; soon we’ll be updating to that version and to Unicode’s locale data from CLDR 21 (both via ICU). The continued rise in use of Unicode makes it even easier to do the processing for the many languages that we cover. Without it, our unified index it would be nearly impossible—it’d be a bit like not being able to convert between the hundreds of currencies in the world; commerce would be, well, difficult. Thanks to Unicode, Google is able to help people find information in almost any language.
Posted by Mark Davis, International Software Architect
Game day can’t come soon enough? Get in the spirit today with the official premiere of Madonna’s “Give Me All Your Luvin” featuring Nicki Minaj and M.I.A. on YouTube. You can find the video on the Madonna YouTube Channel, along with teasers for her halftime show performance.
We’ll also have her video up on the YouTube Ad Blitz Channel pre-game on Sunday, where you’ll be able to view and vote for your favorite commercials.
Today the BSI, Germany’s Federal Office for Information Security, released a best practice guide for Windows users as part of their overall guidelines and recommendations for Cyber Security. Security has always been a core focus of Chrome, so we’re particularly honored to see several of its security benefits recognized in the report:
The browser is the central component for using any online service on the Web and therefore is the most critical attack surface for cyber attacks. Therefore, if possible, you should use a browser with sandbox technology. The browser that currently most consistently implements this protection is Google Chrome (https://www.google.com/chrome). Comparable mechanisms implemented in other browsers are either weaker, or non-existent. By using Google Chrome, in addition to the other mechanisms we mentioned, you can significantly reduce the risk of a successful IT attack.
In addition to Chrome’s sandbox, the guide also highlights the importance of Chrome’s auto-update feature:
Equally positive is the auto-update functionality of Google Chrome, which includes a bundled version of the Adobe Flash Player. By bundling it with Chrome, the Adobe Flash Player will also always be kept up to date.
On the eve of Safer Internet Day, security on the web still faces a variety of challenges. We hope our efforts to improve the security and privacy of our users continue to help make the web a better place.
Posted by Dr. Wieland Holfelder, German Extraordinaire, and Travis McCoy, Security Aficionado
Posted by Adrian Ludwig, Android Security Engineer
We frequently get asked about how we defend Android users from malware and other threats. As the Android platform continues its tremendous growth, people wonder how we can maintain a trustworthy experience with Android Market while preserving the openness that remains a hallmark of our overall approach. We’ve been working on lots of defenses, and they have already made a real and measurable difference for our users’ security. Read more about how we defend against malware in Android Market on the Google Mobile Blog here.
The last year has been a phenomenal one for the Android ecosystem. Device activations grew 250% year-on-year, and the total number of app downloads from Android Market topped 11 billion. As the platform continues to grow, we’re focused on bringing you the best new features and innovations – including in security.
Adding a new layer to Android security Today we’re revealing a service we’ve developed, codenamed Bouncer, which provides automated scanning of Android Market for potentially malicious software without disrupting the user experience of Android Market or requiring developers to go through an application approval process.
The service performs a set of analyses on new applications, applications already in Android Market, and developer accounts. Here’s how it works: once an application is uploaded, the service immediately starts analyzing it for known malware, spyware and trojans. It also looks for behaviors that indicate an application might be misbehaving, and compares it against previously analyzed apps to detect possible red flags. We actually run every application on Google’s cloud infrastructure and simulate how it will run on an Android device to look for hidden, malicious behavior. We also analyze new developer accounts to help prevent malicious and repeat-offending developers from coming back.
Android malware downloads are decreasing The service has been looking for malicious apps in Market for a while now, and between the first and second halves of 2011, we saw a 40% decrease in the number of potentially-malicious downloads from Android Market. This drop occurred at the same time that companies who market and sell anti-malware and security software have been reporting that malicious applications are on the rise. While it’s not possible to prevent bad people from building malware, the most important measurement is whether those bad applications are being installed from Android Market – and we know the rate is declining significantly.
Android makes malware less potent In addition to using new services to help prevent malware, we designed Android from the beginning to make mobile malware less disruptive. In the PC model, malware has more potential to misuse your information. We learned from this approach, designing Android for Internet-connected devices. Some of Android’s core security features are:
Sandboxing: The Android platform uses a technique called “sandboxing” to put virtual walls between applications and other software on the device. So, if you download a malicious application, it can’t access data on other parts of your phone and its potential harm is drastically limited.
Permissions: Android provides a permission system to help you understand the capabilities of the apps you install, and manage your own preferences. That way, if you see a game unnecessarily requests permission to send SMS, for example, you don’t need to install it.
Malware removal: Android is designed to prevent malware from modifying the platform or hiding from you, so it can be easily removed if your device is affected. Android Market also has the capability of remotely removing malware from your phone or tablet, if required.
No security approach is foolproof, and added scrutiny can often lead to important improvements. Our systems are getting better at detecting and eliminating malware every day, and we continue to invite the community to work with us to keep Android safe.
More than two-thirds of Earth is covered by oceans. If you drained all of that water away, you’d see huge basins that have never been explored, an undiscovered planet in our own backyard.
The update covers ocean topography from all over the globe and reveals our most accurate view of the seafloor to date. This new detail comes from spacecraft measurements of bumps and dips in the ocean surface as well as shipboard soundings from surveys carried out by over 40 countries. With this update to ocean terrain data in Google Earth, 15 percent of the seafloor is now available at 1 km resolution.
You may remember a Sun article reporting the discovery of a street grid where it’s believed the lost city of Atlantis would have been located off the West Coast of Africa. The discovery turned out to be a data artifact related to the way data was collected from a ship sailing back and forth to survey an unknown area. This recent seafloor update has been improved to blend better, and “Atlantis” has again disappeared into legend.
Previous terrain data showed what appeared to be a city grid on the ocean floor (left), which was confused for the lost city of Atlantis. Updated data more accurately reflects the seafloor terrain (right).
(Click on image to enlarge)
Another place you’ll notice an improvement in the ocean terrain data is the Mediterranean Sea, particularly south of Cyprus, where the boundary between the Eurasian and Arabian plates is now more clearly defined.
The Mediterranean Sea: before (left) and after (right)
(Click on image to enlarge)
Our last example shows improvements to the land-sea mask along the coastline of Guam. You’ll notice a dramatic improvement in the resolution of both the coastline and neighboring Mariana’s Trench, the deepest trench in the world.
Guam and Mariana’s Trench before (left) and after (right)
(Click on image to enlarge)
In order to make this update possible, our partners at the Scripps Institution of Oceanography at UCSD have curated 30 years of data from more than 8,000 ship cruises and 135 different institutions. If you’d like to get a closer look at these changes, watch our video tour of this new terrain.
We hope you enjoy diving deeper than ever in Google Earth!
Good news for Google Site Search owners who manage multiple Google Site Search engines! You can now share query quota between paid GSS and free Custom Search engines.
This may be useful for you if you pay for a quota with Google Site Search and you’d like to share that quota with your Custom Search engines. To do this, add those extra engines to a “business group” to share the quota.
To add CSEs to a business group, visit the Business Group page of your GSS’s Control Panel. Click Add and you’ll see a list of the free CSEs that you own or that have been shared with you. Select the free CSEs you wish to add and click OK. All group members will immediately share the quota of your GSS.
We hope you’ll find this much easier than trying to estimate your quota needs for each of your individual search engines. To learn more about Business Groups, please visit our help center. Let us know what you think in our discussion forum.
Women make up more than half the global population, but hold fewer than a third of the world’s engineering jobs. In the U.S., female students comprise fewer than 15 percent of all Advanced Placement computer science test takers. Even in high-tech Israel, few girls choose computer science. Not only is this a loss to companies like Google and everyone who benefits from a continually developing web; it’s also a lost opportunity for girls.
Beginning in 2008, a group of female engineers at Google in Israel decided to tackle this problem. We established the “Mind the Gap!” program, aimed at encouraging girls to pursue math, science and technology education. In collaboration with the Israeli National Center for Computer Science Teachers, we began organizing monthly school visits for different groups of girls to the Google office and annual tech conferences at local universities and institutes. The girls learn about computer science and technology and get excited about its applications, as well as have a chance to talk with female engineers in an informal setting and see what the working environment is like for them.
Since we started this program over three years ago, we’ve hosted more than 1,100 teenage girls at our office, and an additional 1,400 girls at three annual conferences held in leading universities. These 2,500 students represent 100 schools from all sectors and from all over the country: Tel Aviv, Haifa, Tira, Beer-Sheva, Jerusalem, Nazareth and more; what they have in common is the potential to become great computer scientists.
The results are encouraging. For instance, some 40 percent of the girls who participated in last year’s conference later chose computer science as a high school major.
We encourage people in other countries, at other companies and in other scientific disciplines to see how they could replicate this program. You can read more at the project site. Currently, we are working with the Google in Education group to expand the program to more offices globally and get even more young women excited about computer science. The difference we can make is real: At one of our first visits three years ago, we met a 10th grade student named Keren who enjoyed math but had never considered computer science as a high school major. Last month, Keren informed us that the visit made such an impact on her, she decided to change her major to computer science. “Talking to women in the field helped me change my mind,” she said.
Posted by Michal Segalov, Software Engineer at Google’s R&D Center, Israel
