ZODB documentation and articles

111 Pages·2017·0.44 MB·English

by  Zope Developer Community

ZODB documentation and articles
Zope Developer Community
Mar 30, 2021

Contents
1 Learningmore 3
2 Downloads 191
3 Communityandcontributing 193
PythonModuleIndex 195
Index 197
i
ii
ZODBdocumentationandarticles
BecauseZODBisanobjectdatabase:
• noseparatelanguagefordatabaseoperations
• verylittleimpactonyourcodetomakeobjectspersistent
• nodatabasemapperthatpartiallyhidesthedatabase.
Usinganobject-relationalmappingisnotlikeusinganobjectdatabase.
• almostnoseambetweencodeanddatabase.
• Relationshipsbetweenobjectsarehandledverynaturally,supportingcomplexobjectgraphswithoutjoins.
CheckouttheTutorial!
ZODBrunsonPython2.7orPython3.4andabove. ItalsorunsonPyPy.
Contents 1
ZODBdocumentationandarticles
2 Contents
1
CHAPTER
Learning more
1.1 Introduction
1.1.1 Transactions
Transactionsmakeprogramseasiertoreasonabout.
Transactionsareatomic Changesmadeinatransactionareeithersavedintheirentiretyornotatall.
This makes error handling a lot easier. If you have an error, you just abort the current transaction. You don’t
havetoworryaboutundoingpreviousdatabasechanges.
Transactionsprovideisolation Transactionsallowmultiplelogicalthreads(threadsorprocesses)toaccessdatabases
andthedatabasepreventsthethreadsfrommakingconflictingchanges.
Thisallowsyoutoscaleyourapplicationacrossmultiplethreads,processesormachineswithouthavingtouse
low-levellockingprimitives.
Youstillhavetodealwithconcurrencyonsomelevel. Fortimestamp-basedsystemslikeZODB,youmayhave
toretryconflictingtransactions. Withlocking-basedsystems,youhavetodealwithpossibledeadlocks.
Transactionsaffectmultipleobjects MostNoSQLdatabasesdon’thavetransactions. Theirnotionsofconsistency
aremuchweaker,typicallyapplyingtosingledocuments. TherecanbegoodreasonstouseNoSQLdatabases
fortheirextremescalability,butotherwise,thinkhardaboutgivingupthebenefitsoftransactions.
ZODBtransactionsupport:
• ACIDtransactionswithsnapshotisolation
• Distributedtransactionsupportusingtwo-phasecommit
ThisallowstransactionstospanmultipleZODBdatabasesandtospanZODBandnon-ZODBdatabases.
3
ZODBdocumentationandarticles
1.1.2 Other notable ZODB features
Databasecachingwithinvalidation Every database connection has a cache that is a consistent partial database
replica. When accessing database objects, data already in the cache is accessed without any database inter-
actions. Whendataaremodified,invalidationsaresenttoclientscausingcachedobjectstobeinvalidated. The
nexttimeinvalidatedobjectsareaccessedthey’llbeloadedfromthedatabase.
Applicationsdon’thavetoinvalidatecacheentries. Thedatabaseinvalidatescacheentriesautomatically.
Pluggablelayeredstorage ZODB has a pluggable storage architecture. This allows a variety of storage schemes
including memory-based, file-based and distributed (client-server) storage. Through storage layering, storage
componentsprovidecompression,encryption,replicationandmore.
Easytesting Becauseapplicationcoderarelyhasdatabaselogic,itcanusuallybeunittestedwithoutadatabase.
ZODB provides in-memory storage implementations as well as copy-on-write layered “demo storage” imple-
mentationsthatmaketestingdatabase-relatedcodeveryeasy.
Garbagecollection Removal of unused objects is automatic, so application developers don’t have to worry about
referentialintegrity.
Binarylargeobjects,Blobs ZODB blobs are database-managed files. This can be especially useful when serving
media. IfyouuseAWS,there’saBlobimplementationthatstoresblobsinS3andcachesthemondisk.
Timetravel ZODBstoragestypicallyaddnewrecordsonwriteandremoveoldrecordson“pack”operations. This
allowslimitedtimetravel,backtothelastpacktime. Thiscanbeveryusefulforforensicanalysis.
1.1.3 When should you use ZODB?
Youwanttofocusonyourapplicationwithoutwritingalotofdatabasecode. ZODBprovideshighlytransparent
persistence.
Yourapplicationhascomplexrelationshipsanddatastructures. Inrelationaldatabasesyouhavetojointablesto
model complex data structures and these joins can be tedious and expensive. You can mitigate this to some
extent in databases likePostgres by using more powerful data types like arrays and JSON columns, but when
relationshipsextendacrossrows,youstillhavetodojoins.
In NoSQL databases, you can model complex data structures with documents, but if you have relationships
across documents, then you have to do joins and join capabilities in NoSQL databases are typically far less
powerfulandtransactionalsemanticstypicallydon’tcrossdocuments,iftheyexistatall.
InZODB,youcanmakeobjectsascomplexasyouwantandcrossobjectrelationshipsarehandledwithPython
objectreferences.
Youaccessdatathroughobjectattributesandmethods. If your primary object access is search, then other
databasetechnologiesmightbeabetterfit.
ZODB has no query language other than Python. It’s primary support for search is through mapping objects
calledBTrees. Peoplehavebuildhigher-levelsearchAPIsontopofZODB.Theseworkwellenoughtosupport
somesearch.
Youreaddataalotmorethanyouwriteit. ZODBcachesaggressively,andifyourworkingsetfits(ormostlyfits)
inmemory,performanceisverygoodbecauseitrarelyhastotouchthedatabaseserver.
Ifyourapplicationisverywriteheavy(e.g. logging),thenyou’rebetteroffusingsomethingelse. Sometimes,
youcanuseadatabasesuitableforheavywritesincombinationwithZODB.
Needtotestlogicthatusesyourdatabase. ZODBhasanumberofstorageimplementations, includinglayeredin-
memoryimplementationsthatmaketestingveryeasy.
Adatabasewithoutanin-memorystorageoptioncanmaketestingverycomplicated.
4 Chapter1. Learningmore
ZODBdocumentationandarticles
1.1.4 When should you not use ZODB?
• Youhaveveryhighwritevolume.
ZODB can commit thousands of transactions per second with suitable storage configuration and without con-
flictingchanges.
Internalsearchindexescanleadtolotsofconflicts,andcanthereforelimitwritecapacity.Ifyouneedhighwrite
volumeandsearchbeyondmappingaccess,considerusingexternalindexes.
• Youneedtousenon-Pythontoolstoaccessyourdatabase.
especiallytoolsdesignedtoworkwithrelationaldatabases
NewtDBaddressestheseissuestoasignificantdegree. Seehttp://newtdb.org.
1.1.5 How does ZODB scale?
Notaswellasmanytechnologies,butsomefairlylargeapplicationshavebeenbuiltonZODB.
At Zope Corporation, several hundred newspaper content-management systems and web sites were hosted using a
multi-database configuration with most data in a main database and a catalog database. The databases had several
hundredgigabytesofordinarydatabaserecordsplusmultipleterabytesofblobdata.
1.1.6 ZODB is mature
ZODBisverymature. Developmentstartedin1996andithasbeenusedinproductioninthousandsofapplications
formanyyears.
ZODBisinheavyuseinthePyramidandPlonecommunitiesandinmanyotherapplications.
1.2 Tutorial
Thistutorialisintendedtoguidedeveloperswithastep-by-stepintroductionofhowtodevelopanapplicationwhich
storesitsdataintheZODB.
1.2.1 Introduction
TosaveapplicationdatainZODB,you’llgenerallydefineclassesthatsubclasspersistent.Persistent:
# account.py
import persistent
class Account(persistent.Persistent):
def __init__(self):
self.balance = 0.0
def deposit(self, amount):
self.balance += amount
def cash(self, amount):
(continuesonnextpage)
1.2. Tutorial 5
ZODBdocumentationandarticles
(continuedfrompreviouspage)
assert amount < self.balance
self.balance -= amount
Thiscodedefinesasimpleclassthatholdsthebalanceofabankaccountandprovidestwomethodstomanipulatethe
balance: depositandcash.
SubclassingPersistentprovidesanumberoffeatures:
• Thedatabasewillautomaticallytrackobjectchangesmadebysettingattributes1.
• Datawillbesavedinitsowndatabaserecord.
Youcansavedatathatdoesn’tsubclassPersistent,butitwillbestoredinthedatabaserecordofwhatever
persistentobjectreferencesit.
• Objectswillhaveuniquepersistentidentity.
Multipleobjectscanrefertothesamepersistentobjectandthey’llcontinuetorefertothesameobjectevenafter
beingsavedandloadedfromthedatabase.
Non-persistent objects are essentially owned by their containing persistent object and if multiple persistent
objectsrefertothesamenon-persistentsubobject,they’ll(eventually)gettheirowncopies.
Notethatweputtheclassinanamedmodule. Classesaren’tstoredintheZODB2.Theyexistonthefilesystemand
theirnames,consistingoftheirclassandmodulenames,arestoredinthedatabase. It’ssometimestemptingtocreate
persistentclassesinscriptsorininteractivesessions,butifyoudo,thentheirmodulenamewillbe'__main__'and
you’llalwayshavetodefinethemthatway.
1.2.2 Installation
BeforebeingabletouseZODBwehavetoinstallit. Acommonwaytodothisiswithpip:
$ pip install ZODB
1.2.3 Creating Databases
WhenaprogramwantstousetheZODBithastoestablishaconnection,likeanyotherdatabase. FortheZODBwe
need3differentparts: astorage,adatabaseandfinallyaconnection:
import ZODB, ZODB.FileStorage
storage = ZODB.FileStorage.FileStorage('mydata.fs')
db = ZODB.DB(storage)
connection = db.open()
root = connection.root
ZODBhasapluggablestorageframework.Thismeansthereareavarietyofstorageimplementationstomeetdifferent
needs, from in-memory databases, to databases stored in local files, to databases on remote database servers, and
specializeddatabasesforcompression,encryption,andsoon. Intheexampleabove,wecreatedadatabasethatstores
itsdatainalocalfile,usingtheFileStorageclass.
1Youcanmanuallymarkanobjectaschangedbysettingits_p_changedattributetoTrue. Youmightdothisifyouupdateasubobject,
suchasastandardPythonlistorset,thatdoesn’tsubclassPersistent.
2Actually,thereissemi-experimentalsupportforstoringclassesinthedatabase,butapplicationsrarelydothis.
6 Chapter1. Learningmore