As a result of moving into a new role, I'm about to embark on a complete re-filing of 5+ years worth of work documents. Typically my digital filing system always ends up completely random, even though I always start out with very good intentions.
Has anyone got any good advice on best practice document hierarchies and naming conventions? Are there any useful tools that can help me speed up this re-filing process?
My documents are all a mix of Word/Excel/Powerpoint/Visio/PDFs and are typically either BAU/Run or Project related.
This discussion has been archived.
No new comments can be posted.
Best Practice Document Management?
|
Log In/Create an Account
| Top
| 41 comments
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(Score: -1, Redundant) by Anonymous Coward on Wednesday July 02 2014, @11:09AM
These aren't the docs you're looking for.
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @04:06PM
Just start e-mailing them to various people who might have a slight connection to them with the notation "for your files". The problem is off your plate and on theirs. As Dilbert's PHB stated, "That's called progress".
(Score: 2) by Nerdfest on Wednesday July 02 2014, @11:36AM
Forget pure hierarchies, they are always limiting. Us a proper piece of document management software that uses tagging.
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @11:59AM
This, thousand times this. All other schemes are inferior.
(Score: 3, Informative) by Sir Garlon on Wednesday July 02 2014, @12:10PM
I'm not disagreeing. Here's essay I liked [shirky.com] about the topic.
Tags have their problems, too, though. Mainly, the way you would tag an item is different from the way I would tag an item. Kind of like using mod points on Soylent: there is loose agreement on what the moderation tags mean, but when the rubber hits the road whether my post is Troll, Flamebait, or just Overrated is very much a matter of individual preference.
I'm not saying don't use tags. I'm saying watch out because there are pitfalls with them, too. They may well be the lesser evil.
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
(Score: 2) by scruffybeard on Wednesday July 02 2014, @01:22PM
You are correct that tagging can be a pain. Having some experience with this, there are a couple of key things to consider (much covered in your link). Is your ontology going to be open or closed, and who will be able to tag the documents? When I setup these kinds of systems I tend to be conservative at first, creating a limited vocabulary, forcing/requesting users to stay within those boundaries, then relax the rules as needed.
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @02:50PM
Could you offer some suggestions? Are there any good Free document management packages you know of?
(Score: 2) by Nerdfest on Thursday July 03 2014, @02:20AM
I've used Nuxeo. It works and uses a standard, open API.
(Score: 1) by bzipitidoo on Wednesday July 02 2014, @05:11PM
Came here to say the same thing. There are better ways than hierarchies.
Now if only we could get rid of the file directory hierarchy and have a file system that uses tagging. Symbolic links do not cut it.
(Score: 2) by Nerdfest on Wednesday July 02 2014, @06:10PM
IO think there are file-systems that do that. You can always use a hierarchy for rough divisions and use tags in addition to that. The path hierarchy is effectively just part of the name. It has some value, but really only in providing a somewhat meaningful unique resource locator for a file.
(Score: 1) by present_arms on Wednesday July 02 2014, @11:37AM
The way I do it is I make directories called /word /spreadsheets /presentation etc under /Documents then from the root directory where the files are sort by extetion (for the none PDF files) drag and drop. Then you have to work out what pdf's come under what category. I don't envy you in your task I can tell you
*yes I used the / as I use linux :)
http://trinity.mypclinuxos.com/
(Score: 1) by EETech1 on Wednesday July 02 2014, @03:43PM
I do this for my documents too, and it works out great!
I have that PDF, or that spreadsheet, I know i am in the right folder, and finding the subfolders is then much easier.
It seems odd to have your data for one project in one folder, and your reports for that project in another folder until you want to compare reports, or grab a copy of some data from a few projects, and it's all in the same place, easy to sort and select in the file manager. It's also easy to see if you had RFQ's that didn't get quoted by looking at the folders or files in each. Want all the orders for a project? right in the orders folder and the project subfolder, not (lost) in the project folder, with a subfolder for each item along with all the other info for that item.
For a business setting, I think it would need a little work though. Upfront definition for each type of item (quotes, rfqs, bids, orders, models, pics, manuals) and creating the proper hierarchy would take a little time.
(Score: 2) by VLM on Wednesday July 02 2014, @11:38AM
Put in more detail, like why the project oriented docs don't just get shoved in the git repo alongside the project. Or in a project-docs repo along side it or some kind of branch in the repo.
As much as I complain about binary thinking, observationally most docs (much like reports, and internal mailing list spam) DO seem to fall in the categories of "I don't want any of this but I don't want to take the blame for getting rid of it, either" and "OMG this is important to me and I know just what to do with it because I really need it why are you messing with 'my' stuff?". So either they've got the wrong guy filing (because you don't already know exactly what to do with it) or you need the most polite and PC and CYA and inclusive and diverse and cooperative system imaginable to none the less eventually send it all to /dev/null or at least get it out of the way. Over decades I really haven't seen much that falls outside those bins.
Its all "OMG thats the most important component data sheet ever for my low noise op amp project its so going in my git repo for project xyz, what were you doing trying to hide it from me over there, I'd be lost without that doc" or "Everybody knows that every recipient of that automated useless report has a rule in outlook to send their personal copy straight to the deleted items, but nobody has the authority or guts to shut it off, so out it goes every day right into everyones trash". And maybe 1 in a 1000 docs falls outside that category.
I worked for a place more than 20 years ago in the early 90s that, I kid you not, used email as their distributed departmental database. Obviously not 50 gig video files but little wiring diagrams and notes in text format in emails sent to everyone and then everyone manually stuck their copy in an email folder for safe keeping. Scales pretty well to one page text files of the list-ish and cheat sheet variety for a couple dozen people and a couple dozen docs.
I worked for another place almost exactly 20 years ago that used the Lotus office suite word processor as their document management system, actually as a customer site master list and they had a secretarial staff person act as a human CRUD interface to do INSERTs UPDATEs SELECTs and custom sorts and joins of the list. By hand. She typed fast, at least, and had some conception of copy -n- paste so it wasn't as slow as you might imagine. So that was our corporate standard database system. Ironically at one of the largest mainframe DB2 database installations in the midwest which everyone thought hilarious.
(Score: 2) by VLM on Wednesday July 02 2014, @11:53AM
Oh let me rephrase this in a way I just thought of thats "supposedly" outside the workplace (but, LOL, we know better)
There exist hoarders. They can't organize stuff themselves. You can outsource "fixing" hoarders but aside from the psychological work, the physical work mostly seems to involve little more than throwing virtually everything into a dumpster. If you sent one of those outsourced hoarder-fixers to my electronics lab in my basement, they'd either have nothing to do and walk away if they're honest or smart, or they'd totally F up my optimized workflow because a shrink has no idea what the workflow looks like (no need to keep the tangle of o-scope probes next to the o-scope if you have no idea what either is or how they are used together) or even basic safety issues (fresh oily wood finishing rags laying all over, well lets make a big pile of them right next to the wood storage pile, what could possibly go wrong?), or they'd screw up valuations of the hardware having no idea what anything is worth so they'd cause me serious economic damage (oh, he uses a multimeter more than he uses the spectrum analyzer... lets just throw the SA in the trash and see how he reacts when he can't find it LOL)
So at work if they're hoarders and you don't care about their psychological state or have an employee assistance plan, just shove all that crap in the dumpster. And if they're not hoarders, you'll be the star of a pitchforks and burning at the stake party if you touch someone else's stuff.
Its almost like someone with more experience who knows whats about to happen is out to get you, setting you up, assuming it was someone elses bright idea.
(Score: 2) by Jaruzel on Wednesday July 02 2014, @12:55PM
Ok, bit more detail on my work flow...
I'm a Solutions Architect/Designer. And other than code snippets to query DBs or convert data, there's no real source code to go with what I work on. I work a lot with vendors, so I get tons of reference docs from them, and I churn out a fair amount of design docs, and visio diagrams myself. Where I work, we don't have a proper DMS, only massively unorganised file shares for teams, and recently (last few years) Sharepoint Document Libraries.
My overall plan is to restructure all my documentation into something more logical, then sync the 'release' documentation to the Sharepoint Libraries. As much as I'd love to have a proper DMS with good search abilities and proper tagging, all I've got is a network folder, and Sharepoint. Sharepoint is a pig of a platform, and whoever at MS decided that IIS+WebDAV was all you needed to make a good DMS for it needs to be taken out and shot. :(
-Jar
This is my opinion, there are many others, but this one is mine.
(Score: 2) by VLM on Wednesday July 02 2014, @02:44PM
git will work anywhere you've got a folder to write to, so its a possibility even without "infrastructure". Someproject/test and Someproject/importer and directory structures like that.
I have stuck things like queries and conversion scripts in git, who knows maybe using those again and especially conversions/importers might go thru some revisions.
I screwed around with a "test driven development" scheme for queries, kinda, once. That was interesting. The fun of the TDD was of course the schema was changing over time as was the use of the table, because the business case was evolving. That sits nice in git too.
(Score: 5, Interesting) by Sir Garlon on Wednesday July 02 2014, @11:55AM
First, since you mentioned Visio and MS Word, I presume you are working in a Windows shop. If your workstation is a Windows box as well, then you can install a thing called an iFilter [markszulc.com] (don't get me started about the stupid name) to enable built-in Windows Search to search PDFs. That should be quite handy if you didn't already know about it.
Second, a tip. What you are actually trying to do is to create a taxonomy. There is a profession called "taxonomist." Originally it required a master's degree in library science; lately junior-level taxonomists have been coming from the IT world and learning on the job. Doesn't matter. It's a profession, and it's probably not your profession. My point is that creating your own filing system is like doing your own taxes. Anybody can do it on a small scale, with some effort. The bigger the scale, the more time and difficulty you can expect. So my advice is, approach the problem with humility and realize the problem you've observed, of your filing systems coming out "completely random," is typical. Make sure your boss knows that the task you've taken on would probably require weeks of full-time work by an expert. My taxonomist friend recommends a book if you are ready to start developing expertise: The Accidental Taxonomist [hedden-information.com]. I have not got around to reading it so this is second-hand advice.
I'm not a taxonimist but I know I need one myself. I maintain a Wiki in my spare time and the categories are a train wreck because I made them. ;-) Strangely enough, my taxonomist friend declined the opportunity to fix my problems for free. Imagine that!
[Sir Garlon] is the marvellest knight that is now living, for he destroyeth many good knights, for he goeth invisible.
(Score: 2) by Jaruzel on Wednesday July 02 2014, @01:00PM
Yes, totally agreed on the taxonomy thing. My better half is a Chartered Librarian, with a degree in Library Management, and she's got uber taxonomy skills (plus many others related to data management) - however I can't ask her to sort my work docs out, that'd be just rude - she's got her own workload to deal with :)
Actually, I'm pretty sure we've got that book sitting in a pile of other data management books somewhere in the house... I should go see if I can find it.
-Jar
This is my opinion, there are many others, but this one is mine.
(Score: 1, Insightful) by Anonymous Coward on Wednesday July 02 2014, @12:03PM
That way you're much more likely to be able to access the content a few years from now.
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @12:07PM
I can still access 15+ year old Excel and Word documents in the latest version of Office. In what universe will one not be able to access the content after only "a few years from now"?
(Score: 3, Informative) by mtrycz on Wednesday July 02 2014, @12:32PM
When the document format provider goes out of business. It's not like it hasn't happened before.
In capitalist America, ads view YOU!
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @01:10PM
Office will still run regardless of whether or not Microsoft stays in business.
(Score: 2, Insightful) by gargoyle on Wednesday July 02 2014, @01:31PM
New installs of office won't, because the licensing servers won't respond.
In fact that is a risk even if Microsoft is still going as a corporate entity
(Score: 4, Informative) by MrGuy on Wednesday July 02 2014, @12:05PM
"Has anyone got any good advice on best practice document hierarchies and naming conventions?"
Yes. Don't rely on document hierarchies and naming conventions to solve this problem.
Use an actual document management system with attributes, descriptions, and tagging. Your filename will never be sufficiently descriptive for every purpose under which you'd be looking for a document, and no fixed hierarchy will ever be sufficient.
Here's an example - let's say one type of thing you were documenting was vacation photos. If I were setting up a hierarchy, I might use something like /photos/vacations/YYYY/destination/day/filename.jpg This seems like a reasonable organizing scheme, until I realize that the photo I really want to send to my friend is one I took in Jamaica - it's of me and Tom at sunset.
Now the hierarchy is a hinderance, especially if I've been to Jamaica more than once - I now need to separately search /photos/vacations/2003/Jamaica/ and /photos/vacations/2006/Jamaica/ directories, and I have to trawl through each day separately. I suppose you could say that I should have made the destination ABOVE the year, but a.) that won't necessarily be optimal for a DIFFERENT search, and b.) still means I have to trawl a bunch of directories one-at-a-time. And there's no way to get to "all my photos with Tom" if I didn't organize that way, which might be "the thing I actually want." ("Wait - was it Jamaica, or was it Cancun?")
Also, even "good" filenames won't help you. Maybe you'll get lucky and the photo will be named "Me_and_Tom_at_sunset.jpg" But it might be named "Me_and_Tom_at_Montego_Bay.jpg" or "Sunset_at_Montego.jpg" or "Me_and_Tom_at_Montego_Bay.jpg" There are strong limitations to how much information you can encode in a readable filename, and the information you chose at the time might not be the information that's relevant at retrieval.
An actual document management system lets you define some attributes you consider relevant (who, where, when, time of day, etc), and give you space for a brief text description (way better than a short filename and actually searchable) and some free-form tagging if you need it. Let the system handle where the file goes and what to call it. When you want your file back, you tell it the attributes you know, and it will find the likely ones for you.
(Score: 2, Insightful) by hendrikboom on Wednesday July 02 2014, @02:27PM
And you'd ideally make sure your document management software is somewhat open-source, so it won't get pulled out from under your feet when an incompatible upgraded happens, or when a corporation-quake destroys your software supplier.
-- hendrik
(Score: 3, Funny) by Dunbal on Wednesday July 02 2014, @12:36PM
I have some best practice documents. Somewhere. Hang on, let me remember where I put them...
Seriously organization is something you learn at a young age. By the time you're an adult if you haven't realized that you save time by investing a little time and let laziness and procrastination get the upper hand, then there's probably no hope for you - you already have the bad habits. What makes you think that if you had the document you seek you'd actually a) bother to read it and b) actually put it in practice?
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @12:50PM
First I put all the unsorted documentation in a single folder. Avoid the urge to classify everything upfront, except maybe in broad categories (programming, financial, etc). Then I can search for the *content* I need when I need it. For that I use Google Desktop, that still works great.
As I find the documents I need, I sort them (or link them) according to the need. For example, if I need document X.pdf for project Y because it describes something required I link it from /projects/y/requirements
This is a poor man tagging system that has served me well for years.
(Score: 1, Redundant) by wonkey_monkey on Wednesday July 02 2014, @01:35PM
Let me see... I've got it written down somewhere...
systemd is Roko's Basilisk
(Score: 2, Interesting) by hendrikboom on Wednesday July 02 2014, @02:16PM
Hierarchical classifications work if your aim is to place documents somewhere physical. Each document gets a specific place to be, where you can find it, say, on a bookshelf. This is what libraries use.
That said, you may have noticed that libraries also (used to) have a card index, because books are often about more than one thing.
tags have become more useful with the advent of computers. You can assign multiple tags to a document, and use a computer to find entries with particular combinations of tags. It's important to have a specific, standard set of tags, with cross-references (such as "see" or "see-also" links) for synonyms or near-synonyms.
You might look, say, at the Bliss library classification index at http://www.blissclassification.org.uk/ [blissclassification.org.uk] for a particular set of tags. (this URL seems to be only intermittently available -- try it a few tines before giving up) You might also, but you'd probably want more specialized ones for your specific application.
The Bliss classification is a faceted classification system -- a book is described by a number of facets; each is like a tag, a property of the book. To make it a hierarchical system, it has a particular prescribed order in which the tags have to be concatenated to make a classification string, but if you have a computer that can search a data base, you won't need this part.
You might also have a look at the colon classification, invented by R.S. Ranganathan http://en.wikipedia.org/wiki/S._R._Ranganathan [wikipedia.org]. Specifically, look at his Prolegomena to Library Classification http://worldcat.org/oclc/003586800 [worldcat.org] , which incisively analyses the issues involved in library classification.
Again, you are probably more interested in the organisation of the facets than how to string them together into a hierarchical classification. Remember, these systems were invented before computers were easily available in every library.
The Bliss classification indexes are available on-line, and they are being maintained.
That said, you probably don't care all that much for the alphanumerical codes these systems use, but are interested in facets they describe. Use them, and add others as appropriate. Don't forget to make an organised list of the facets you invent, so you can find them later when you need them again.
There's also the Open Directory Project, which may also give you a few ideas: http://www.dmoz.org/ [dmoz.org]
-- hendrik
(Score: 1) by hendrikboom on Wednesday July 02 2014, @05:27PM
(Score: 0) by Anonymous Coward on Thursday July 03 2014, @10:46AM
You might also have a look at the colon classification
I use that with certain people, I tell them to shove their document up their colon
(Score: 3, Funny) by elf on Wednesday July 02 2014, @02:55PM
This is not relevant to the storey at all, but it made me laugh. Mod me down :)
I saw the headline of this article but what caught my eye was the category in the top right, it was "Answer" and it was underlined (looked like a link). I was't 100% "there" and I clicked it thinking it had the answer to the question (I thought that was clever).
I feel dumb and stupid but it made me chuckle!
(Score: 2, Informative) by Uncle_Al on Wednesday July 02 2014, @03:13PM
I think someone who works as an archivist would be a better person to talk to
than a librarian. They are not the same profession, and archivists deal with
collections of unstructured content all the time.
"Everything is Miscellaneous"
http://www.everythingismiscellaneous.com/ [everythingismiscellaneous.com]
Is a good place to start when dealing with the problems of taxonomy in unstructured content.
(Score: 2) by elf on Wednesday July 02 2014, @04:02PM
Any one of the above can help out, I like confluence a lot as you attach documents of all the types you want. It will also index all those files and make the items searchable. The order of storing is less important then because you can search and find what you want pretty easily.
All the items I mentioned have a cost element so it could depend on your budget.
If you have no budget and just want to store things in a folder structure for organising its quite hard to give advice. If its project related you could go by Project-section of project, if its related to reports you could just do report - etc.
(Score: 2) by Open4D on Wednesday July 02 2014, @09:25PM
I suppose the specialist software you mentioned lets people get away from the limitations of filesystem-based document management. Perhaps a poor man's version* of that could be to give every file a UUID and dump them all into one single flat folder in the filesystem. Then use a database of the UUIDs to do the actual management. (Postgres? Or maybe just a spreadsheet?)
And searching could possibly be handled by something like this "enterprise search platform": http://lucene.apache.org/solr/ [apache.org] ? (Though I've never tried it.)
* - or a "free man's version" if the reason for not using Sharepoint et al. is more about software freedom than price.
(Score: 1) by pgc on Wednesday July 02 2014, @09:53PM
Confluence is a great wiki. But it is awful for archiving documents. The fact that you can do it, doesn't make it a good idea.
(Score: 2) by jackb_guppy on Wednesday July 02 2014, @05:02PM
Put them all on public facing website. Let Google index them. Done. :)
Actually, at one point Goggle had a server to index your local network. Knew a few law firms looked at it help them.
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @05:08PM
Google is NOT your friend.
http://www.telegraph.co.uk/technology/google/10940358/Google-buys-Songza-for-a-reported-39m.html [telegraph.co.uk]
"It has a huge database of information on listening habits"
Total Information Awareness goes "yum!"
(Score: 0) by Anonymous Coward on Wednesday July 02 2014, @09:09PM
"Goggle had a server to index your local network"
Yet another product that wasn't shiny enough to keep their attention.
(Score: 2) by Jaruzel on Wednesday July 02 2014, @09:10PM
Actually, at one point Goggle had a server to index your local network. Knew a few law firms looked at it help them.
They still do, it's called a Google Search Appliance [google.com].
I evaluated the original 'Google in a box' appliance a few years ago for an employer - although it worked well, it didn't support samba/smb file shares, and needed everything to be HTTP facing (they've since resolved that restriction). Microsoft's FAST search software ended up stealing a lot of the features shortly afterwards.
-Jar
This is my opinion, there are many others, but this one is mine.
(Score: 2) by egcagrac0 on Thursday July 03 2014, @04:45AM
The shop I work at has a reasonably short retention policy - about a year.
If the project is open, fine, keep the docs.
If the project is closed for at least a year, discard the docs.
If your customers are paying you to keep the documents longer, then that should have been already worked out during the quote phase of the project.