Saturday, April 22, 2006

Trusted TLDs - What are Trusted Top Level Domains?

Do .edu and .gov links have, in general, a higher value and weighting factor in the overall scheme of things?

I was recently involved in a very exciting and lengthy discussion centered around a simple question from someone who asked...

Is there any evidence that having inbound .edu and .gov links are better links and count more than other links?

An immediate two character reply of "No" was presented as the answer to the question.

I, on the other hand, had something more to say on the subject. And it sort of went like this...

If someone gave me the choice of having a .com, .edu or .gov inbound link and all things were equal, I'd take the .edu and .gov over the .com any day. From my understanding, they do have a different value in the overall scheme of things. They are trusted TLDs and not everyone can get one like the .com, .net, .org TLDs. A link from a .edu or .gov site is going to have more weight than one from a .com site in general. Now, if the .com is an authority site, then things get a little tricky in determining which would be better. But don't worry about it. If you can get legitimate .edu and .gov links, definitely get them.

After that reply, I came under heavy artillery fire. I was surrounded. Trapped and nowhere to go. Just the type of environment I really excel in. So, I ended up putting together a synopsis detailing my personal experience and that of others when working within a Trusted TLD space such as .edu.

Much of the information I responded with is based on a few years of experience in working with an authority .edu TLD. I also have experience in other TLD sectors that are of a Trusted nature. I've read and have assimilated various patents that have been applied for and approved by the U.S. Government in regards to search.

The specific patent I used for my supporting documentation was filed by Google on 2003 December 31. It is referred to as United States Patent Application 20050071741 and is titled . . .

Information Retrieval Based on Historical Data
http://www.google.com/search?&q=20050071741

What Makes a TLD (Top Level Domain) Trusted

I'm going to focus on the two TLDs that were referenced which are the .edu and .gov extensions. Let's take a close look at the restrictions that govern the .edu and .gov TLDs. First we need to establish what makes these TLDs Trusted in the sense that they are legitimate and quality TLDs "out of the box" and provide a little more value in the overall picture.

The .GOV TLD

I'm going to save you the time of researching the .gov restrictions and tell you that the .gov TLD is an authority and is Trusted out of the box. You are welcome to perform the hours of research that I did to effectively determine that. You can start here...

.GOV Registration
http://www.dotgov.gov/

The .EDU TLD

On July 25, 2003, Amendment 6 to the Cooperative Agreement clarified and changed several important policies applicable to the .edu top-level domain:
  • Names in the .edu top-level domain, regardless of when issued, may not be transferred in any way by the Registrant to any other entity.
  • Transferring" includes selling, trading, leasing, assigning, or any other means of transferring.
  • Names in the .edu top-level domain that are not grandfathered must reasonably represent the name of the Registrant, may not be deployed to identify any organization other than the Registrant, and may not be "generic names."
  • Registrants in violation of these policies will be notified by EDUCAUSE in writing. Violations not corrected in 45 days will result in removal of the registration of the .edu name.
  • EDUCAUSE will conduct a six-month study of .edu names not in compliance with current policies and will make recommendations to the Department of Commerce with respect to bringing such registrations into compliance.

Amendment 9

  • On October 21, 2004, EDUCAUSE and the Department of Commerce signed Amendment 9 to the Cooperative Agreement, updating the .edu eligibility requirements as follows:
  • Modify the .EDU eligibility requirements to comprise two categories of organizations. Category 1 would be labeled "Accredited Institutions"; Category 2 would be labeled "System/District Offices".
  • Category 1 would contain the current eligibility criteria (postsecondary institutions accredited by one of the agencies on the Department of Education list of National Recognized Accrediting Agencies).
  • Category 2 would contain this text: "University system offices, state coordinating offices or boards, community college district offices, or equivalent entities located within the United States which have as their principal activity the management and governance of a collection of 'Accredited Institutions' that themselves meet the eligibility criteria for .EDU.
  • Each entity, in order to qualify under this category, must be commissioned, established, or recognized by a state, local or national government to perform this management and governance activity."

Amendment 11

On February 2, 2006, EDUCAUSE and the Department of Commerce signed Amendment 11 to the Cooperative Agreement. Amendment 11 authorizes EDUCAUSE to:

  • Continue administration of the .edu Internet domain through September 30, 2011.
  • Implement a procedure addressing violations of the prohibition against transferring .edu domains from one entity to another.
  • Assess an annual $40.00 administration fee to recoup the expenses of managing the domain. Administrative contacts for each .edu domain will receive a series of communications about the fee starting in early May.

If you have any doubt at this point that the above restrictions do not indicate a sign of legitimacy, quality and trust, please refer to the below links for more detailed information.

EDUCAUSE
http://www.educause.edu/

.EDU Policy Information
http://www.educause.edu/edudomain/policy.asp

.EDU Frequently Asked Questions
http://www.educause.edu/edudomain/faq.asp

Extracting Information from the Patent

Patents are revealing. In this particular instance, we have Google filing for a patent that covers just about every thing you could think of related to the quality of search. If you are really interested in how the search engines review documents, reading one or more of the patents filed will give you a level of knowledge that few in this industry are willing to pursue. Too many SEOs are focused on "what they see" in their browsers while severely neglecting the foundation that supports what they see.

Okay, from this point forward it is going to get somewhat technical. I'm going to be pulling quotes from the patent and also from the guidelines of the .edu TLD. I will not focus on the .gov TLD as they are both (.edu and .gov) classified as Trusted TLDs. All official documentation will be quoted. My notes will not be quoted and appear directly below each reference from the patent itself.

What is claimed is:

A method for scoring a document, comprising: identifying a document; obtaining one or more types of history data associated with the document; and generating a score for the document based on the one or more types of history data.

The below excerpts from the patent are directly related to the discussion of .edu and .gov links being Trusted out of the box. They "naturally" have a higher value in the overall scheme of things.

28. The method of claim 26, wherein the weight assigned to a link is based on at least one of how much a document containing the link is trusted, how authoritative a document containing the link is, and a freshness of a document containing the link.

Item #28 indicates that weight is assigned to the document based on Trust. I'd like for you to keep that word Trust at the forefront of your thoughts while reading through this stuff. This is really like watching paint dry for many of us but, to truly understand what that paint is made of, you need to peel back the layers.

39. The method of claim 38, wherein the scoring the document includes: determining whether the domain associated with the document is legitimate, and scoring the document based, at least in part, on whether the domain associated with the document is legitimate.

Note the word "legitimate" above. Due the restrictions in place for .gov and .edu TLDs, they are legitimate out of the box.

49. The method of claim 1, wherein the one or more types of history data includes information relating to linkage of independent peers; and wherein the generating a score includes: determining a growth in a number of independent peers that include the document, and scoring the document based, at least in part, on the number of independent peers.

Note the term "independent peers". Typically .edu and .gov TLDs are going to have a high number of Trusted TLDs linking to them. These are independent peers and provide an additional level of trust in the overall equation.

58. The method of claim 57, further comprising: determining longevity of the linkage data; deriving an indication of content update for a linking document providing the linkage data; and adjusting the ranking of the linked document based on the longevity of the linkage data and the indication of content update for the linking document.

Note the word "longevity". Typically .edu and .gov TLDs have instant longevity. Rarely will you find .edu or .gov TLDs expiring. This fact gives these two TLDs instant longevity.

59. The method of claim 58, wherein the adjusting the ranking includes penalizing the ranking if the longevity indicates a short life for the linkage data and boosting the ranking if the longevity indicates a long life for the linkage data.

Note the word "longevity" again. But this time the reference is to a penalty incurred if the longevity indicates a short life for the linkage data.

Alternatively, if the content of a document changes such that it differs significantly from the anchor text associated with its back links, then the domain associated with the document may have changed significantly (completely) from a previous incarnation. This may occur when a domain expires and a different party purchases the domain. Because anchor text is often considered to be part of the document to which its associated link points, the domain may show up in search results for queries that are no longer on topic. This is an undesirable result.

Note the term "domain expires". The .edu TLD is non-transferable. Transferring includes selling, trading, leasing, assigning, or any other means of transferring. This implies a level of Trust in that the domain is not going to be used in an "Expired Domains" purchase.

Please keep in mind that the same process can be applied to most TLDs. But, out of the box, .gov and .edu domains have restrictions on their use that make them Trusted TLDs which is implied in a few patents out there. The .com, .net, .org, .etc. TLDs do not have that Trust factor out of the box because they are open to abuse on a far greater scale than the .gov and .edu TLDs.

As we dig deeper into the patent, and please, do read it from top to bottom, we are able to uncover some very interesting correlations with what the patent states and what our own experiences (meaning all of us) prove to us.

[0074] Links may be weighted in other ways. For example, links may be weighted based on how much the documents containing the links are trusted (e.g., government documents can be given high trust). Links may also, or alternatively, be weighted based on how authoritative the documents containing the links are (e.g., authoritative documents may be determined in a manner similar to that described in U.S. Pat. No. 6,285,999).

Note that the patent implies that government documents can be given high trust.

[0097] According to an implementation consistent with the principles of the invention, information relating to a domain associated with a document may be used to generate (or alter) a score associated with the document. For example, search engine 125 may monitor information relating to how a document is hosted within a computer network (e.g., the Internet, an intranet or other network or database of documents) and use this information to score the document.

Note the word "hosted" and the term "computer network". Typically .gov and .edu TLDs are going to be hosted on a quality network. One that is void of the abuse that may take place on another server not hosting these types of domains.

[0098] Individuals who attempt to deceive (spam) search engines often use throwaway or "doorway" domains and attempt to obtain as much traffic as possible before being caught. Information regarding the legitimacy of the domains may be used by search engine 125 when scoring the documents associated with these domains.

Note the word "legitimacy" again, but this time in the context of throwaway domains. That doesn't happen in the .gov and .edu TLD sectors. Another level of Trust.

[0099] Certain signals may be used to distinguish between illegitimate and legitimate domains. For example, domains can be renewed up to a period of 10 years. Valuable (legitimate) domains are often paid for several years in advance, while doorway (illegitimate) domains rarely are used for more than a year. Therefore, the date when a domain expires in the future can be used as a factor in predicting the legitimacy of a domain and, thus, the documents associated therewith.

Note the words "legitimate" and "illegitimate". You're going to see those words quite a bit when digging through various patents related to search. In the above example, domain expirations may be used in the overall equation in determining the quality of a domain. While I wouldn't put too much weight into this one, it could be one of many "signals" present that are used to determine legitimacy of a domain.

[0100] Also, or alternatively, the domain name server (DNS) record for a domain may be monitored to predict whether a domain is legitimate. The DNS record contains details of who registered the domain, administrative and technical addresses, and the addresses of name servers (i.e., servers that resolve the domain name into an IP address). By analyzing this data over time for a domain, illegitimate domains may be identified.

Note the word "legitimate" again. .gov and .edu TLDs have very strict and governed policies in regards to POCs (Points of Contact), whois information, technical information, etc. Another level of Trust.

[0101] Also, or alternatively, the age, or other information, regarding a name server associated with a domain may be used to predict the legitimacy of the domain. A "good" name server may have a mix of different domains from different registrars and have a history of hosting those domains, while a "bad" name server might host mainly pornography or doorway domains, domains with commercial words (a common indicator of spam), or primarily bulk domains from a single registrar, or might be brand new.

Note the term "name server". Ah-ha, you probably didn't even think about that one eh? Hosting environment is imperative in establishing a Trusted TLD. This applies to all TLDs and not just the naturally Trusted ones like .edu and .gov.

[0102] In summary, search engine 125 may generate (or alter) a score associated with a document based, at least in part, on information relating to a legitimacy of a domain associated with the document.

.edu and .gov are "legitimate" domains out of the box. They can therefore be considered Trusted TLDs which are most likely weighted accordingly and don't incur the same scrutiny other generic TLDs (.com, .net, etc.) are subjected to.

While the patent may not be evidence to support my claims, it does shed some light on what I have personally experienced and continue to experience with all TLDs under my management. Others have shared similar experiences. This is not something that "has to be proven" although many would like it in writing. It's a natural occurrence and one that has been confirmed by many in the industry who are working within those Trusted TLD spaces.