Différences

Ci-dessous, les différences entre deux révisions de la page.

--- en:temoignages:moteur_orange [2011/06/27 20:07] – daamien
+++ en:temoignages:moteur_orange [2011/07/06 21:51] – daamien
@@ Ligne 1: / Ligne 1: @@
-====== France Telecom managing a 24 Terabytes database with PostgreSQL  ======
+====== France Telecom's Search Engine based on a 24 TB database powered by PostgreSQL  ======
+//In the Following text, Severine Aubry and Robin POUGET, who are respectively Project Manage and Database Manager at France Telecom, demonstrate their use of PostgreSQL for a major tool of they company.//
+France Telecom has build a web search engine called "Le Moteur" (http://www.lemoteur.fr/). Behind this engine lies a back office that includes all that the machinery needed to store the keywords contained by web sites (URLs), analyzing them, indexing them, etc.. This application is highly critical because as it determines the quality of the results produced by the search engine.
+This back office must be refreshed in 24hours/24 and 6days/7.
+ In other words, we can tolerate a non-updated weekly.
+The "back office" of the search was conducted between 2001 and 2002, only with PostgreSQL. At that time, the project began with version 7.4. Today the project is based on PostgreSQL 8.2. The "back office" has, over the years, of course, been some improvements and fixes.
+In detail, the engine is composed of a "crawler", whose charge is to browse internet with a list of URLs and key thousands of reference sites. It follows links automatically.
+The charge of "crawler" is to retrieve URLs and associated content.
+This data is then processed through several scripts written in TCL.
+Data is stored in a schema with a few thousand tables. The partitioning is based on technology developed internally, based on hash key type.
+The thrust of the project are from the beginning to have a solution:
+    stable
+    robust
+    very maintainable
+    scalable, capable of cashing a soaring volume and therefore
+    free from the problems of disk space, or the number of servers needed PostgreSQL
+Some figures:
+billion and tuples are distributed
+machines that are home to 800 servers PostgreSQL on Linux,
+    for a total volume of 24 terabytes
+    Note that PostgreSQL is not only running on these machines, there are also applications
+The machines are spread over three geographical sites, with a logical division, called "software blocks."
+There are data export this set to other data servers for various uses.
+This results in high flexibility of the entire application: if a server has to fall, there is no interruption of service because the data it contains are redundant. There is no more dependence between the elements.
+There has been in the history of the project a few minor issues that were fixed by the community, whose support is deemed "effective":
+    problems of fragmentation of data, linked to massive UPDATE. This has been fixed with the new features of PostgreSQL over the years (remember the project over 10 years with PostgreSQL!)
+    Version 8 has corrected any concerns in terms of memory management;
+    VACUUM FULL's almost ancient history. We went into this project from March to April VACUUM FULL to 1 year only.
+In conclusion, PostgreSQL gives satisfaction here over 10 years. The few problems encountered were all treated with the utmost effectiveness of the community. New versions of PostgreSQL have brought more or solutions to these problems, or improvements or simplifications.
+[Interview by Jean-Paul Argudo, March-June 2011]
 Séverine AUBRY and Robin POUGET témoignent de l'utilisation de PostgreSQL pour un outil de France Télécom.

Suivez-Nous !

Utilisateur

Page

Rechercher

Différences