This project was completed on 2017-08-22. The rest of this page is retained only for archival purposes
As of mid 2016, we have a new Developer Guide (aka Dev Guide) written with MkDocs and intended to replace most of the content within the Developer Documentation section of this wiki.
As of early 2017, we are actively migrating pages from the wiki to the Developer Guide.
This migration project will be completed when all of the following are true:
- When you look at the wiki's page tree, the "Develop" item will no longer exist
- All wiki pages currently listed under "Develop" with either be (a) migrated to the Developer Guide (and eventually removed from the wiki), or (b) kept on the wiki long-term with a new spot in the wiki's page tree (not under "Develop").
- URLs that point to existing pages within the wiki, will redirect to the corresponding page in the Developer Guide. This process is already happening, as we migrate pages. Here is an example redirect.
Target project completion date: Fall, 2017
Sean Madsen is leading this project. Find him on Mattermost if you have questions.
Process for migrating other areas of the wiki
This project only covers the Developer Documentation section of the wiki.
There are many other wiki pages outside the scope of this project. A notably separate project is documentation on administering CiviCRM installations (e.g. installation, upgrades, etc.) and for this content we have plans to create a separate Administrator Guide. (But first we'd like to finish the Dev Guide project.)
Even still, there are other areas of the wiki not encompassed by the topics of "development" or "administration". As we continue to move content out of the wiki the fate of this other, more miscellaneous content is currently unknown.
How to edit existing documentation during the migration
First a foremost: go forth and edit! No need to hold back. We have systems in place to perform this migration in parallel with the ongoing maintenance of existing documentation.
Most wiki edits will be captured during page migration
If you find a wiki page that you would like to improve, then please do so! Your ability simply to view the page on the wiki means that the page has not been migrated – once pages are migrated, redirects are automatically put in place to bump visitors over to the new page in the Developer Guide. So any changes you make to that not-yet-migrated page will almost certainly be captured as part of its migration.
Wiki edits not captured in migration fall into the "safety net"
There are a few edge cases when edits to the wiki are not captured by normal page migration:
- Timing edge case: When the edit is made within the (hopefully short) time window between the point when page migration begins and ends. For example, let's say Marge copies wiki content to markdown on Monday, in preparation for migration. Then on Tuesday Ed edits the wiki page. Then on Wednesday Marge completes her cleanup and migration. Ed's edits will not be reflected in the final page in the Developer Guide.
- Status edge case: When the edit brings an existing page from a state of lacking content worthy of migration to now having some content worth migrating, and when this happens after the page has been evaluated in the spreadsheet (described in the migration process section below).
- New page edge case: When the edit adds a new page
Safety net process
This is a manual process (but quick) and performed regularly by Sean Madsen.
It was last performed on: 2017-08-22T12:02-06:00
- Look at the recent updates to the wiki, one by one, since the "last performed date"
- Look at each edit using "view change" when necessary. Immediately disregard edits that meet any of the following criteria:
- pages outside of CRMDOC/Develop
- Look for edge cases, as described above.
- For a timing edge case, manually port the changes made in the wiki into the Developer Guide by looking at the diff in the wiki.
- For a status edge case, update the status in the wiki pages spreadsheet.
- For a new page edge case, add the page to the wiki pages spreadsheet.
- Update the "last performed date".
When to edit the Developer Guide
If you get redirected to the Developer Guide after viewing a wiki page you seek to edit, then please read about contributing to documentation in the Developer Guide so that you can make your edit there. If you're short on time and don't want to dig into editing content in an unfamiliar system just yet, then you may submit an issue describing changes that should be made,
If you have brand new content to add (e.g. new page), then please add it to the Developer Guide if possible because that will save other people the time needed to migrate it – but even if you add it to the wiki, it will still get caught by the safety net described above.
MkDocs is the platform on which we have chosen to focus our documentation efforts, going forward. It has already been used to create the User Guide, with great success. It's a python-based tool that generates a static website from content written in markdown files. The files are then kept in GitHub for collaborative editing.
Why migrate to MkDocs?
The wiki has served us well. But MkDocs offers the following improvements:
Differentiation between the wiki and the Developer Guide:
Yes, we are moving a lot of pages off the wiki. But this doesn't mean the wiki will cease to be useful. Here is a rubric to clarify the intended purpose of the wiki, going forward.
- Developer Guide
- Official documentation for existing and aspiring developers.
- Explains development tools and methodologies, generally.
- Examples are given minimally, and only when necessary to explain larger theories.
- Topics are covered comprehensively.
- Can be read linearly. Cumulative skills ordered sequentially for optimal reader comprehension.
- Content held to high standards.
- Written with MkDocs
- Wiki – Reqs, Specs, & Recipes
- A place for requirements and specifications to be stored and reviewed.
- A place for recipes – i.e. specific and detailed examples of accomplishing a task that will be uncommon to most developers
- Little thought given to presenting content in a didactic order.
- Minimal content standards.
- Written with Confluence.
Here's the grand vision for the project:
- Pick pages on-by-one to move from the wiki to the Dev Guide
- In progress
- This process is being managed by this spreadsheet of wiki pages (in the same document as the above spreadsheet). Read more details about this spreadsheet by choosing the "README" tab at the bottom.
- The process of choosing pages to migrate can be done in parallel by many people.
- Where disagreements arise about the fate of a specific wiki page, discussion can happen on Mattermost. A page status even exists "to-discuss".
- Migrate pages one-by-one
- In progress
- For a given page to be migrated, see the steps below "how to migrate one page"
- Separate pages can be migrated in parallel by many people. Use the "Assignee" column in the spreadsheet is to prevent someone else from stepping on your toes as you begin the work to migrate a page or set of pages.
- The migration process of specific pages can begin before the status of all pages is decided.
- Re-organize wiki after all pages are migrated
- Not yet started
- To better usher-in the "specs & recipes" era of the wiki and allow it to thrive, we'd like to do the following
- Delete the wiki pages that have been migrated to the Dev Guide
- Re-organize pages that will remain on the wiki.
- The process for this is to-be-determined.
- --- At this point the project is deemed "Complete" ---
- Eventually, move the Developer Guide repository into the CiviCRM core repository so that future changes to CiviCRM core can be committed alongside documentation updates
- Not yet started
How to migrate one wiki page to the Dev Guide
- Learn how to edit the Dev Guide.
- Use the spreadsheet of wiki pages to pick a page to migrate.
- Assign the page to yourself in the spreadsheet by writing your name in the "Assignee" column.
- Identify a target location in the Dev Guide
- Keep in mind that we want cumulative skills to be presented sequentially.
- If you need to create a new page in the Dev Guide, use the mkdocs.yml file to add it to the navigation menu.
- Copy-paste content into the file(s) in the Dev Guide
- Keep your markdown clean by following the standards on the Markdown page within the Dev Guide
- Optionally, follow instructions below for automatically converting a wiki page to markdown
- Add a redirect from the wiki page to the Dev Guide page (see instructions below)
- Make it final
- Commit your changes and submit a pull request
- Update the status of the wiki page in the spreadsheet to "done-migrated"
- (You don't need to wait until your PR is merged to update this status)
How to use
webpage2md to convert one wiki page to markdown
- Ensure you have the following packages installed (which are all available via apt-get and brew):
curl pandoc html-xml-utils
- Get the wrapper script,
git clone https://github.com/seanmadsen/webpage2md
- Then run like this:
./webpage2md 'https://wiki.civicrm.org/confluence/display/CRMDOC/The+codebase' > codebase.md
- You can have it automatically pick a filename for you and save the file in your current directory
./webpage2md -a 'https://wiki.civicrm.org/confluence/display/CRMDOC/The+codebase'
- See all the options
- If you pull pages from CRM (instead of CRMDOC), you need to set the -c option as follows to change how it picks content from the page
./webpage2md -c 'div#main-content' 'https://wiki.civicrm.org/confluence/display/CRM/Book+style+guide' > style.md
How to redirect one wiki page to the Dev Guide
As an example, to redirect:
add the following line in
Commit the change to this txt file along with your commit that adds the page. Then when your PR is merged, the redirect will be automatically added to the wiki.
How to view an old wiki page after it has been redirected
If this wiki page has been redirected (as described above),
then add an extra slash after CRMDOC as follows, and you'll be able to view the page: