extensions

Tripal User's Meeting 2018/09/07 Tripal Community Meeting Notes

  1. Tripal v3.0 full release imminent!  -- just working on some last minute fixes and testing.

  2. Future of Tripal Discussion

    1. Andrew Farmer:

      1. Would like to continue to integrate with Tripal community and co-ordinate and co-develop and use what others develop.

      2. Faced with the problem: don't want to build websites that are tied into a single framework technology.

    2. Ethy Cannon

      1. Some sites have very little resources (few developers).  But don't have time to learn Drupal 8/Tripal 3.  

      2. Lacking developer capacity makes people reliant on Tripal community to help build tools.  

      3. Worried Tripal is getting locked into old technologies in being tied with Drupal

      4. Main developers of Tripal are extremely skilled with Drupal but for those without those skills it can be very challenging.  Especially or adding customizations.

      5. Drupal should be easier to integrate with non Drupal technology.

      6. Worried that being tied with a 3rd party framework you are at the mercy of what those developers do?

        1. Bradford:  We are always stuck with some framework

        2. Andrew:  We want to adopt what was developed and piece together what works.  Not only don’t want to be tied to a single framework, they often don’t have that luxury

        3. Ethy:  Having a website built inside of a monolithic framework that doesn't easily support integration with other tools is a problem.

        4. Meg:  Working with a single framework is it helps manage permissions, and easier to manage cross site-searching.  Having multiple components all strung together complicates systems administration.

          1. Andrew: containers may be able to help with this.

          2. Lacey:  also want to provide a consistent user experience. I believe this is key for users being able to learn/navigate a given resource. The user shouldn’t need to care about there being different pieces underneath the website -searching and navigating should be seamless

          3. Andrew:  a consistent user experience may not be as important for users who have to use multiple sites, because switching across sites causes inconsistent user experience.

    3. Sook asks Andrew about his experience

      1. Phylotree viewer and expression work that integrates with Tripal

      2. Genome Context Viewer: JavaScript / Python -- developers are talented in those skill sets.  He doesn't think it's a good use of their time to relearn Drupal/PHP when they are skilled in technologies that can already work.

    4. Bradford: What is Tripal?

      1. Chado + Drupal = Tripal… To make Tripal platform agnostic may be more than our group can handle. He thinks if we change it would be a framework change.

      2. Ethy:  Chado is a difficult schema to work with.  Tripal should be driving Chado and defining how data is mapped into Chado.  She's interested in an approach that allows Tripal to provide functions to Chado but allow for other technologies (including Drupal)

      3. Meg:  Anthony wrote a Python loader for Chado

      4. Bradford:  Is understanding Ethy's desire to decouple the Chado integration from the Drupal code.   He agrees that Tripal does bring a common interface to Chado. The perception is that Tripal is an API to Chado.  

      5. Meg:  It is doable to separate out Chado code as a "library".  Do we want a "headless" Drupal or a lower-level library that decouples Tripal from Drupal.

      6. Bradford:  Are we trying to move away from Chado?  (consensus is no).

      7. Andrew:  A service layer may be good enough.  

      8. Bradford:  object with this may be that if he writes a module he can now only share the Tripal part and not the Drupal part.  Decoupling the code may limit how much we can share.  Would decoupling create more work to maintain?

        1. Ethy:  if the framework uses the tool then you take the framework with it.  If you bring in lots of different tools with you have to manage all those tools.

        2. Lacey:  It's harder to collaborate with others if everyone has different approaches with different frameworks.

    5. Do Tripal  v3 web services provide the ability to decouple?

      1. Stephen: too slow at this point, need to improve performance

      2. Lacey: can’t post data through web services

    6. Thoughts:

      1. Lacey: wants to stay with Drupal and does not want to re-write site in another platform.  But she is warming to the idea of that Tripal could be used without needing to work in Drupal.  That may help with learning curves because it keeps people out of Drupal-land. But it is not a full decoupling.  

        1. Ethy:  will be difficult to move away from PHP (she would like to learn a new language, however).  

      2. Stephen:  In a conversation with Dorrie she indicated she wants Tripal to stay with Drupal.  

        1. Sook:  would be too difficult to leave Drupal because of all the infrastructure already built.

    7. Stephens proposal:

      1. Break Tripal in ½: that related to data storage into a PHP library and then maintain the Drupal module for the front-facing and integration

      2. We need our community! We need to support flexibility to keep our community.

      3. But we also need the common language and framework for when we are between or low in funding. We need to share modules and development effort.

    8. Lacey: focus how to make Tripal to truly integrate 3rd Party software.  (i.e JBrowse, Django, etc.) How to have a consistent interface.

      1. Ethy agrees (150%).

    9. Lacey: regarding switching languages or frameworks just to chase what is most popular should be considered heavily. We cannot switch from language to language over time.

    10. Lacey's Proposal:  Take existing classes for Tripal v3 and use those to form the basis of a PHP Tripal library.  Then when we move to Drupal 8 it may make the upgrade easier. It can still provide a middle layer that folks can learn and avoid the Drupal layer.  Lacey loves what Drupal allows you to do despite its difficulties.

    11. Lacey:  would a Tripal PHP library help with development in your group, Andrew?

      1. Andrew:  Language is important.  What is built into the language is important.  The Tripal and Chado loaders don't behave the same way.  Anthony has had issues with the PHP loader and so he created the Python loader.  His group is not excited about PHP. He's not sure a PHP library would increase the utilization of it.  

      2. Ethy:  If we do a middle or sub-layer it should be in an object oriented language and PHP is not object oriented. It's object-capable.   It can cause trouble. She thinks it's not safe.

    12. Discussion about using python for the middle layer

      1. Ethy: feels it’s sufficiently close to PHP to make learning it easier for PHP guru

      2. Lacey: worried about how do we actually integrate between python and PHP. If this is done with web services, I worry about performance. Is there an alternative to make the integration of php

        1. Looks like PHP+Python is not a new concept, for example: https://www.csh.rit.edu/~jon/projects/pip/

      3. Stephen: provides closer integration with python groups (e.g. Anthony’s group, Andrew’s group). Closer integration perhaps with the galaxy community. There might be some help from Andrew’s group for such an endeavour

      4. Bradford: perhaps we should be talking about wrapping the PHP for use by python users. Similar to Anthony’s work

        1. Reason being: the core tripal people use and enjoy PHP

        2. We shouldn't switch languages to appeal to people on a “if you build it they will come” philosophy

      5. Main counterpoint: PHP is not great with object oriented.

        1. PHP does not enforce standard object oriented rules: too loosely typed. This causes issues with inheritance

      6. Steve:  maybe not so much a focus on object oriented but more with the monolithic approach to Drupal where it's not clear “what to do where” in development.

      7. Bradford:  MVC

      8. Andrew: has heard good things about newer PHP frameworks.

      9. Lacey would prefer that we stick with PHP because it is easiest to maintain what we already have. If we feel OO capabilities are sufficient then let's stick with PHP.

      10. Bradford: he feels the scientific community is more interested in learning Python.

        1. Specifically they are biologists and can use python down the road for scientific computing, less so for PHP.  Similar to Andrew’s comment below.

      11. Andrew:  A small non-profit organization and so folks float between different tasks that are not just focused on web development.  Python seems to cross multiple domains.

        1. Ethy:  informatics students are typically working in Python.  There is no reason an informatics student shouldn't be writing modules for Tripal and if that's their language we should accommodate.

    13. Bradford:  If we are leaning towards a middle layer Tripal. Does really satisfy the issues we're facing?

    14. Ethy: whatever we decide to do, we should do a proof of concept to see if it works before we jump in wholesale.  Lacey especially agrees.

      1. Stephen:  proof of concept:  move GFF to middle layer

      2. Ethy:  have these type of activities at our annual hackathon.

      3. Lacey:  better POC would be modeling entities/fields in the middle layer and get Drupal to render it.  Expose only one type of data type and it's fields.

      4. Let's do POC with Drupal 8.

 

What would the Proof of Concept look like:

  1. Tripal Middle Layer

    1. Language: Python.

    2. Abstracts communication with data storage

      1. Provides a Chado layer

    3. Provides some sort of service for use with Drupal

    4. Use Case:

      1. Provides Entities/Fields for a gene.

      2. Easier for researchers who might prefer Python to write modules.

      3. Supports those who want to stay in Drupal without impeding their ability to maintain their sites.

  2. Tripal Drupal 8 Module

    1. Consumes the services from the Tripal layer.

    2. A PHP wrapper for existing API

 

Timeline

Sept 17th: call to form a group of folks who want to work on POC.

November 15th:  revisit and see how it's going.

January 2019 PAG hackathon would be to implement Tripal 4 based on results on the proof of concept.
Tripal User's Meeting 2018/10/05 Tripal Community Meeting Notes

Reminders & Announcements

  • New meeting time!  Tuesdays!  Same time/regularity.

  • Next meeting: November 6th. (US Election Day)

    • Check if we want to postpone it a week.

 

 

  • Stay connected: Follow us on twitter, join our slack space, watch the github repository to see issues/discussions, join the GMOD email list

Discussion Topics (Open Agenda)

  • Official Tripal 3 release!

    • Some parts of guide still need to be updated (functional annotation)

      • Gerard - Upgrade went OK(ish?)  A couple of errors at the end of the install.

        • Please share them on the queue!

        • Note we have continuous integration set up to upgrade a barebones tripal 2 site to tripal 3, so we can at least confirm that is behaving itself.

Im k

  • Nathan

  • Valentin - add fields…

    • Feature_cvterm….

    • X_cvterm

      • The annotations field does this.

      • But isnt perfect for all needs:

      • Example use case: for stock table, with a CV ‘stock_status’ having the cvterms ‘active’, ‘lost’, ‘archived’,’..., it would be nice to define a new field “Chado cvterm” the way “Chado property” is, where you can select the CV to use for the available cvterms to select.

        • A custom field could be the answer

        • Alternatively a generic linker field that is an alternative to the existing annotation field.  It would behave more like the chado property field, allowing the admin to create a field using hte cvterm, but the term/value is stored in _cvterm instead of prop.

Docker images:
Tripal User's Meeting 2018/11/13 Tripal Community Meeting Notes

  1. PAG Events

    1. Dec 3rd.  Last day to update abstract text

    2. Will let speakers know if they will be invited to give a talk at the workshop at PAG.

    3. Thursday-Friday Jan 10-11th.  Codefest + User's Meeting

      1. Goal: more coding than discussion

      2. Current Topics:

        1. Topic 1: Tripal v4 Python Proof of Concept.  Leader: Stephen Ficklin

        2. Topic 2: Tripal + BrAPI Integration.  Leader: Valentin Guignon

        3. Topic 3: Chado Updates.  Leader: TBD

        4. Other Suggestions Welcome

          1. I’d also be happy to continue the discussion on gene pages with a larger group (Monica P).

      3. Sign up Sheet: https://docs.google.com/spreadsheets/d/1agEK8S4yxphTOC7cojvsVjTxsPH5PenvtHDBDz5XotE/edit?usp=sharing

      4. Plan for remote attendance at the sessions.  

        1. We need to coordinate with T&C for better internet and power.   The Codefest committee will get that organized.

    4. Sunday Jan 13.  4:10pm Tripal Workshop at PAG

    5. Monday Jan 14th: AgBioData Meeting.  Garden Salon 2 , 10-12am.

  2. User's Meeting Dec 4 to prepare for Codefest.

    1. Group leaders will connect with their groups to organize tasks.  Watch for multiple online conference links.

  3. Reminder: Slack Workspace. https://join.slack.com/t/tripal-project/shared_invite/enQtNDExNDU1NzUxOTI3LTNlOTg1MTNjZGQ1NjRmNGQxMWRjOGVjN2IwMjQyOTc2ODZlYWM2OTQxM2Y1ZWM1MmFkNWJhOGEwN2RkZjdkNGU

  4. Consolidate documentation for core and extension modules

    1. Use Read the Docs  https://github.com/tripal/tripal/tree/7.x-3.x/docs

    2. We will link to extension modules with RTD docs in the Tripal User's Guide

    3. Two options

      1. Use Sphinx to import docs from other projects

      2. Link to repo or documentation, or a one sentence description with a link out.

  5. If something comes to your mind that you want to discuss add an issue!  Ask to have it tagged with 'Discuss in User's Meeting' tag.

  6. New Module Announcements (please add your own):

    1. Tripal Headquarters (Bradford)

      1. Provide an administration layer so users can submit content to your site pending approval before it goes into Chado.

      2. Allows you to define chado-specific permissions.

    2. Tripal Apollo (Bradford)

      1. User management service for Apollo: connect your apollo server(s) to chado records and auto-sign users up for apollo-side permissions.

        1. Gensass has their own apollo integration with some potential overlap of interest.

        2. Should Monica/Bradford reach out?

    3. JBrowse Management Module (Abdullah)

      1. manage jBrowse completely from Tripal to choose an organism from your Chado database and create a JBrowse instance.

      2. Provides forms for adding tracks without having to deal with perl or command-line.

      3. Different from https://github.com/tripal/tripal_jbrowse which adds a field on an entity page.

    4. Coexpression networks module (Stephen)

      1. Not quite ready to be released, but please get in touch if you would be interested.

  7. Gene Page Improvement (Bradford)

    1. https://github.com/tripal/tripal/issues/732

    2. Perhaps we should reach out to a collaborator in computer science to help direct us on human computer interaction.

    3. Book recommended by grant reviewer: https://www.springer.com/us/book/9781493903771

    4. Quick solution

      1. ONE: just write one field (thats what i started doing)

        1. Problem: its really hard to display so much info in one field.

      2. TWO: Make providing the child info the responsibility of each individua lfield.

      3. Asking each field to recursively travel the feature_relationship table on its own, and not sharing wiht other fields, seems like a major problem to BC/AA.

      4. Therefore, you also provide a feature_relationship lookup.  This would be in the form of featurepath (think: cvtermpath), OR in the form of a cached, shared child relationship lookup.

    5. Things to consider

      1. Fields go to web services

      2. Page and web services should avoid providing different information.

    6. Look at Intermine pages for examples, TAIR, Ensemble

  8. Code Coverage Integration (Bradford)

    1. https://github.com/tripal/tripal/issues/714

  9. How to improve and increase collaborative coding (Stephen)

    1. https://github.com/tripal/tripal/issues/540

    2. What are you working on this week and are you open to collaboration if another group is interested?

  10. Advisory committee. Who should be on it?  What is its role? Let's formalize by PAG

    1. https://github.com/tripal/tripal/issues/344
Tripal User's Meeting 2019/01/11 @ PAG in San Diego Tripal Community Meeting Notes

  • Summary of CodeFest coding groups work/accomplishments

    • Chado Group

      • New release/migration paradigm for how changes are made to Chado and new releases are bundled/created, using flyway.

      • Continuous integration checking for integrity of schema.

      • New (platform agnostic) Docker container for just Chado.

      • Governance discussion.  Moving from benevolent dictator model -> PMC type model.

      • Documentation migrating from the wiki into readthedocs style guide.

    • Tv4 Group

      • In an effort to provide an alternative to PHP, we intended to create a set of microservices in Python to replace the functionality provided by Tripal. During this meeting we created 1 example microservice API using Flask which can be queried to get information out of a database, regardless of whether that database is CHADO

      • As another trial, we created a service using what’s referred to as a microframework in PHP called Lumen. The service uses docker compose to pull in all needed services such as MariaDB, PHP-FPM and Nginx. The purpose is to explore alternatives to Drupal without having to leave PHP.

      • Drupal 8 module.

        • We are thinking about keeping this a do-able process for smaller sites ie that they wouldnt have to learn and implement the microservices framework themselves.

      • What about permissions?  Dorrie is interested in node_access and how usable it would be in Drupal 8.

      • Will we have to set up Docker to set up production Tripal 4 ?  

        • no

 

    • BrAPI Group

      • A new git branch was created to handle changes to be Tripal 3 compatible. It’s the 7.x-3.x branch.

      • BrAPI module installed on multiple sites (6), started to configure the module for those sites

      • Many issue were found as we run the module on different environments, most of them were fixed

      • Started to implement new calls but a tutorial is requested

      • Valentin started writing a tutorial to describe how to implement calls

      • Lacey started a “read-the-doc” for BrAPI module

      • We also discussed site-specific needs and how to handle them

        • Restricting access to certain data subsets...

  • Tripal Governance Discussion Group Report

    • https://docs.google.com/document/d/1tOjaMHBbiJjx5zdN59wUXZpMP31giMOWQ2uY-dx6Rz0

    • Who should be involved in the oversight committee?

      • Experts in various web/tech fields

      • PMC (Project Management Committee)

      • Stakeholders who are users

      • Industry stakeholders

      • Actually organized with an “internal” oversight committee, and then on top of that, an external review board with some of the above participants.

 

    • Every 3rd/4th dev call, make it a steering committee call?

    • Opinion/Whitepaper for Tripal?  What is it and why should you be excited, how do we keep Tripal relevant in the future….

      • Who will be in charge/write it?

      • Good item to discuss at first steering committee meeting?

    • Community-wide funding….

      • Need to collect better usage statistics

        • Shawna is working on a registration module that would report some tracking info.

        • Will report to tripal.info, which is D8 which ahs restful API as part of the core… need a hosting solution

        • So reporting will be included in future tripal 3 releases as an opt-outable module

      • Need to have a good plan for future (which the white paper planning paper would accomplish)

      • Need to do a better job on the visibility of Tripal, growing Tripal

      • Larger international cooperative funding....

      • Should we go directly to program managers for funding.

      • Need a functional steering committee sooner rather than later?

 

  • Tripal Modules Rating System

  • How to encourage people to contribute to core

    • People unsure of how to write sharable code…

    • Is training the issue?

    • So do we need Tripal training?  Or Drupal training?

    • Can we bring community members in and have them offer their services to help with training or answer questions.

    • Could compile a list of people with expertise and pull them into conversations

    • But perhaps would be better if those people would self-volunteer within the issue queue.

  • How many people are still actively developing Tripal 2.

    • Several users are still in Tripal 2.  Specific issues include views functionality and install issues.

    • Plans NOT to convert to Tripal 3 until programming experience

  • Sofia is hiring for a remote tripal developer :P

  • Not upgrading from Drupal 7…

    • “Backdrop” of Drupal 7 as a backup plan? https://backdropcms.org

    • Need to be aware of security issues, need to be aware of the security issues with core releases no longer updated.

 

  • Dinner plans
Tripal User's Meeting 2019/02/05 Tripal Community Meeting Notes

This is an open agenda meeting.

 

  • Thank You

    • PAG Codefest Committee:  Stephen, Ethy, Meg

    • PAG Workshop:  Bradford, Valentin, Meg

  • Reminders

  • Open Topics

    • Email from Maria Martin, lead software development in UniProt at EMBL-EBI:

      • "I was wondering if representation and visualization of proteins is something of interest for Tripal. My team is developing web components for sharing visualizations amongst other activities."

      • Perhaps a topic at our March meeting.

      • Meg will follow up to see if she wants to present.

      • Something like tools on? http://pdb101.rcsb.org/

  • GitHub Issues

    • Below issues are tagged as “Discuss at next User’s meeting”.  We don’t need to go through them all necessarily.

 

  • Tripal Advisory Committee https://github.com/tripal/tripal/issues/786

    • Draft document here (thanks Meg) https://docs.google.com/document/d/12bgwGcoH_od6hyle-Aa4USrc-DS8cZ9bbJvfB3jQGyQ/edit  (See the section titled Science Advisory Committee towards the bottom)

    • Summary from Meg:

        • An additional layer of governance with members not on the PMC

        • PMC would be for steering the code

        • SAC would act in an advisory role only

          • provides leadership, guidance and coordination for Tripal

          • Should include extension developers, site developers, external topic experts

          • Thinking about Tripal from a broader perspective: overall priorities, direction, timelines, funding strategies, recruitment, data best practices, community support, etc.

      • Ethy: Two year membership should be easily renewed if the member is willing

        • Create a minimum of external and internal members

      • From Wikipedia: 8 members is a maximum

        • Also good to start small

      • Start with an internal board and work up to external members

        • Also it might be harder to get external members since they have less of a stake in Tripal

      • Primary meeting in January at PAG with teleconference option

        • Quarterly meeting during the developer call?

          • Allows more of a role by the community and allows the community to guide the SAC

      • Majority based voting since it is a recommendation

      • Posted publicly for transparency. Hopefully this doesn’t discourage ppl to discuss openly.

      • Emergency meetings can be called by the chair

      • Very important to pick the first chair!

      • Strongly encourage all SAC agenda items posted to the github issue queue!

      • Sook commented that it might be help to have multiple layers. In rosaceae there is a chair, vice chair and secretary. They move up in the ranks so only the secretary is voted on. The secretary takes notes and this provides a training model.

      • Ethy comments that perhaps the chair should delegate to ensure that ppl with the right personalities take on the various tasks.

        • Phrase the chair responsibilities as “they will ensure that”

      • Could send out a request for rejection to the mailing list and then set a timeline. Important to get this started

      • Add proposed members to the request

      • Chair should probably not be on the PMC

      • Chair should be someone who has been active in our User's meetings in the past.

      • Acting SAC:

        • Acting Chair Nominations: Ethy *, Acting Co-Chair Meg.  Until April 2nd.

        • PMC will talk about who to recommend to add to SAC

        • Duties:

          • Send out a request for objection about the current approach over general email list or issue queue.

          • Setup the internal stakeholder group

            • SAC Member: Send out a personal email directly to all the PIs or folks who lead the website development to ask who is interested.

            • Internal stakeholders should be a mixed group of PIs and lead site developers.  When sending out emails to PIs be clear that involvement could be the PI or their lead developer.

            • perhaps a GMOD person on the list.

          • Provide update at March Users Meeting.

 

 

 

 

  • Backdrop.  If time (i’d prioritize this over many of the github issues let's make sure we get to it BC): proposed plan for T4 and beyond.

    • Concern: small development team stretched too far

    • Concern: seem to be going in too many directions

    • Create an issue on GitHub for it.
Tripal User's Meeting 2019/03/12 Tripal Community Meeting Notes

Tripal Governance

 

Why? We are getting pretty big and are receiving many contributions (great!). With these contributions we want to ensure the integrity of Tripal is maintained, while also ensuring everyone has a say in the direction of the project :-) We also want to coordinate funding and ensure we are working collaboratively and as a unified community.

 

See Current Governance document: https://docs.google.com/document/d/1iRGmZzKnAm8c5Q9yYLiTctSJaRqnT6Bm0eix1uzoTxw/edit?usp=sharing

 

Discussion

  • How does this document get accepted/rejected?

    • Send this out to the whole community

    • If no rejections within 1 week then we will consider it accepted

    • Submit a PR to the Tripal documentation and leave it open a week

  • How to amend

    • Amendments should be done through PR? We will leave this up to the board once it’s formed

  • How is the chair chosen?

    • Elected within TSC. Voting by entire community probably not necessary (but may be needed to select initial chair).

  • Do you like this proposal?

    • No serious objections from call attendees

  • Start in on the specific discussions on the document

    • What is Tripal Core? Multiple Repositories (essential) or simply the github/tripal/tripal repository

      • Bradford feels github/tripal/tripal is the core

      • Resolved as github/tripal/tripal

      • If we would like to go beyond tripal/tripal then a docker instance would be a good way to formalize this structure

    • What is the authority of the steering committee?

      • What happens when the TPMC unanimously feels against the decision of the Steering committee

      • Ethy feels the TPMC should be the authority on fine-grained code-related details where the steering committee should have more authority on long term goals and steering

      • Meg errs on the side of let the TPMC be the final say since they are the ones who have to write the code

      • Resolved: the advisory board acts in an advisory committee

    • Responses kept to github?

      • Bradford agrees  we won’t be able to get everyone to respond on github in timely email

      • Ethy: copy emails into the github discussion. We need to ensure people have a github account

      • Bradford: if they don’t watch the issue queue but are subscribed to the steering committee group then they we will get the emails notifications

      • Meg: steering committee won’t be watching the full issue queue but hopefully the chair will facilitate the use of github/slack. Having email/slack/github will get overwhelming so we need to consolidate so the chair will need to ensure the channels of communication are working

    • Too much responsibility on the chair

      • Delegate! And elect a vice-chair if you need

      • The first steering committee can revisit this once they see how it’s working

      • Ethy: we don’t want to over-specify roles within the steering committee to ensure that those with the best skills for a job are able to step in. This works best through delegation

    • Internal stakeholder meeting first and then an external stakeholder meeting after

      • We want to see what the committee consists of

      • Lacey: we have to remember that our internal members may end up more high level then some of our external members (e.g. an external technology expert is interested in low level code decisions whereas an internal PI may not)

    • Proposal development:

      • Please look at the document! This is a new discussion and we need/want input!

 

Action Items:

  • Ethy to clean up any redundant information DONE

  • Lacey to make a PR to add governance document to the the official Tripal documentation

  • Next meeting have an official vote for members/chair
Tripal User's Meeting 2019/05/07 Tripal Community Meeting Notes

 

Agenda

  • Tripal Releases

    • Tripal v3.2 coming out soon (May or June)

      • more frequent minor releases of Tripal 3 to follow

    • Tripal v4 (drupal 8 upgrade) is in the planning stage

      • Let us know if you have time available to help! We can all learn Drupal 8 together ;-P

    • Tripal v5 is still up for discussion as a possible decoupling of Drupal and addition of microservices

    • Drupal 8 introduction

  • The new Tripal Steering Committee: Ethy Cannon [ekcannon@iastate.edu], Stephen Ficklin [stephen.ficklin@wsu.edu], Valentin Guignon [v.guignon@cgiar.org], Dorrie Main [dorrie@wsu.edu], Monica Polechau [monica.poelchau@ars.usda.gov] , Meg Staton [mestato@gmail.com], Manuel Ruiz [manuel.ruiz@cirad.fr], Jill Wegrzyn [jill.wegrzyn@uconn.edu], Lacey Sanderson [las166@mail.usask.ca]

  • Open Agenda!

    • Anything you’re working on to share with the community? New modules?

    • New Modules!

      • TSeq (Tripal Sequence Similarity Search) (Developer: Sean, TreeGenes)

        • Diamond & BLAST jobs on existing site data/user-submitted data on external application servers with Tripal Remote Job module

        • Progress - fully functional, adding more features/bugfixes

        • Next milestone: Use Tripal Galaxy rather than Tripal Remote Job

      • TCP (Tripal Contact Profile) (Developer: Rish, TreeGenes)

        • Creates user profiles, data is submitted into contact and contactprop chado tables

        • Progress: Live on Mimubase.org and DEV, to push to TG LIVE

        • Latest Milestone: integrate with any Tripal 3 site

      • TPPS (Tripal Plant Popgen Submit pipeline) (Developer: Peter, TreeGenes)

        • Submit association genetics/landscape genomics study data in a uniform way to Tripal sites

        • Progress - Live in TreeGenes site (links)

        • Upcoming Milestone: Fix portability issues and issues with interfacing with CartograTree

      • CartograTree (Developer: Nic, Rish, Ronald, TreeGenes)

        • Map-based module for analyzing/visualizing phenotypic, genotypic, and environmental study data

        • Progress - In Progress on Tripal Galaxy integration

        • Upcoming Milestones: Complete basic analysis aspects by August 2019

      • OrthoQuery (Developer: Sumaira, TreeGenes)

        • Orthologous gene family analysis (OrthoFinder), uses Tripal Galaxy API

        • Process, annotate, and visualize gene families in a phylogenetic context

        • Progress: Completion of visualization, Completion of storing Unigenes in TreeGenes, Developing the tripal-galaxy workflow for each use cases.

        • Upcoming Milestones: Updated Release Date for August 2019 include implementation of galaxy workflows in the module for all use cases and enabling the visualization to query functional annotations through the database.  

      • Modified gFACs (Gene Filtering Analysis and Conversion) (Developer: Alyssa, TreeGenes)

        • Uses Tripal Galaxy to run a custom workflow featuring gFACs where GTF/GFF files can be filtered and formatted for Jbrowse loads.

        • Filtering focuses on structural and functional aspects of gene models that are problematic following gene prediction.

        • Progress - New project for Summer 2019

        • Upcoming milestones: September 2019 - Complete module.  Currently exists as an independent application: https://gfacs.readthedocs.io/en/latest/index.html

      • EnTAP (Eukaryotic Non-model Transcriptome Annotation Pipeline)

        • Uses Tripal Galaxy to run sequence similarity, protein domain, gene ontology, and gene family assignment.  Specifically features a phylogenetic and contaminant filtering component.

        • Progress - Early Stages of Development.  Currently exists as an independent application: https://entap.readthedocs.io/en/latest/

      • HQ (https://github.com/statonlab/tripal_hq)

        • Tripal HQ provides a user-contributed content control center and administrative toolbox for your Tripal site. This means that users are able to create whatever Chado content you'd like them, but withhold inserting it into the database until someone has approved it.

      • EUtils (https://github.com/NAL-i5K/tripal_eutils)

        • This module connects to the NCBI EUtils API to load in accessions for the Assembly, BioProject, and BioSample databases. Primary, as well as linked, records are loaded into Chado.

    • Job openings in Staton lab! 2!

    • Questions? Concerns?

      • Andrew Farmer brings up their Phylotree module: is there overlay with orthoquery and room for collaboration

      • Andrew Farmer has functionality for on the fly synteny comparisons which TreeGenes may be interested in (GenomeContextViewer). Uses a Chado backend with a services layer above that. Single Page application

      • Meg brings up performance concerns with the Tripal 3 fields.

      • Should you leapfrog over Tripal 3 if you still have a Tripal 2 site?
Tripal User's Meeting 2019/09/10 Tripal Community Meeting Notes

 

Agenda

  1. October's meeting is for the advisory committee.

    • everyone is invited.

  2. Updates from the PMC

    • Tripal v4 progress

      1. Straight port to Drupal 8 (no new features)

      2. Jump on Github to contribute!

        1. https://github.com/tripal/t4d8

        2. Note this is a temporary repo to keep things organized, will be merged back in to normal core repo later

      3. Backwards compatible unless Drupal forces us not to be.

    • Questions about core development?

  3. Updates from the TAC (Meg Staton, Chair)

    • Organizing Grant efforts

      1. Who is writing grants, will it impact core, can it help fund core development?

      2. Perhaps explore some Microsoft Research funding.

      3. Meg perhaps lead a CSSI

      4. USDA FACT possibility for US folks.

    • Next month:  what do we see is the impetus for our community to grow:  "big ideas". Where should we go? Where should it be in 5 years.

      1. use cases

      2. portable version:  give individual labs the ability to use Tripal more easily and connect to established community databases & reproducible workflows.

      3. Jill can start a brainstorming document.

  4. PAG. 

    • Organizing committee:  Lacey & Jill

    • Please reach out to the committee about the Hackathon if you have ideas on how to improve.

      1. How can folks join remotely? 

        1. We will be working on making this more seamless

  5. Training Galaxy Module (Stephen Ficklin, WSU)

    • Documentation: https://tripal-galaxy.readthedocs.io/en/latest/index.html 

    • Github: https://github.com/tripal/tripal_galaxy

    • The purpose of this module is to integrate Tripal and Galaxy

    • Previously the community would create a module which would write the glue for a specific workflow but with more complex workflows and since Galaxy is a moving target this resulted in a lot of developer time

    • The Tripal Galaxy module takes care of a lot of the overhead to reduce developer input

    • When using the Tripal Galaxy module make sure to use a compatible version of Galaxy

    • Supports both automatic webform creation and custom applications submitting to and receiving from Galaxy

    • Uses blend4php which maps the galaxy API over to PHP: github.com/galaxyproject/blend4php

    • See the documentation for how to setup: https://tripal-galaxy.readthedocs.io/en/latest/?badge=latest

      1. You also need to make sure the Tripal: upload files permission is set for users using the workflows

    • Automatic webform creation will create a Tripal/Drupal form on your Tripal site which hides Galaxy behind it

      1. The keeps a consistent user experience

      2. It also allows you to provide custom help to your specific community for an already set workflow

    • A nice thing about using Galaxy is that we can use existing tools and save a lot of time over creating our own workflow management system. It is also a great tool with a large dedicated developer community

    • Allows you to upload site-wide files which any user can utilize in their workflows

    • If you don’t have a dedicated galaxy instance you can configure the module to use https://usegalaxy.org

      1. Create a specific user for your Tripal site rather then using a personal account

    • Even with automatic webform generation you can still tweak a variety of settings including the workflow title, instructions to the user, “About this step” field for each step, and information for each tool.

      1. The user is able to control specific options of the galaxy workflow as they step through the Tripal Galaxy webform. Everything that they can set in the galaxy workflow will be displayed in the webform! Some options will be preset and not changable to make submission easier for the user (e.g. files from previous steps)

      2. Once you submit the last step you are given a full summary of all options picked. This is great for users to keep track of what was done for a publication

    • Tripal > User file management (provided by core Tripal) allows administrators to set site-wide and per user file quotas which apply to files uploaded as part of Tripal Galaxy.

      1. Workflow forms will tell the user where they are in their specific quota and prevent them from uploading files over their quota

    • Files are removed automatically after 60 days (configurable).

    • Any workflows submitted through the automatic form will be submitted as Tripal Jobs and if you use the Tripal Daemon the job will be launched automatically.

      1. The Tripal Daemon also has settings to allow parallel execution of jobs and you can control how many threads are used.

    • Tripal Galaxy provides an API for checking for workflow status

      1. Users are emailed when their job is submitted and when it completes. These email messages can be configured on a per-workflow basis.

      2. There is also an “Analyses” tab on the user page which allows users to check their submission details, retrieve their results and look at any error messages if the analyses did not complete successfully

    • Meg: we have trouble scaling to if someone submits a very large file that will take 3mo. To run

      1. Jill/Sean: Galaxy does configure with your job management system which will allow to configure allocations for cores and memory

        1. You can configure this at the workflow or tool level through Galaxy

      2. Stephen: Tripal Galaxy only provides regulation of storage size, everything else is on the galaxy side

    • Power Applications (option 2 for using Tripal Galaxy)

      1. This is used for Cartogratree

      2. Tripal Galaxy provides an API which allows a more powerful application which then submits to galaxy behind the scenes

      3. This is well documented on the Tripal Galaxy RTD documentation: https://tripal-galaxy.readthedocs.io/en/latest/API/api.html

Tripal User's Meeting 2019/11/05 Tripal Community Meeting Notes

 

Agenda

Community Highlights

Please add anything you would like to highlight about your Tripal-based projects: publications, new extension modules, new collaborations, newly funded grants, project update, etc.  We won't talk about these in the meetings but may tweet them from the Tripal twitter account if you approve!

Name

Highlight Description

Can we tweet it? Yes/No

Jill/TreeGenes

Review paper on cyberinfrastructure for forest trees (highlights Tripal, AgBioData, etc) :https://onlinelibrary.wiley.com/doi/full/10.1111/eva.12860 

Yes

Agenda

  1. Messages from the PMC (Lacey Sanderson, Bradford Condon, Stephen Ficklin)

    • Tripal v4 Update

  2. Messages from the TAC Chair (Meg Staton)

    • None at this time.

  3. PAG  Workshop & Hackathon Committee (Lacey Sanderson, Jill Wegrzyn, Chris Childers)

    • New Abstract Deadline: November 8

      1. There are still open time slots!

    • Suggestions for Hackathon Working Groups

      1. Tripal 4 (upgrade of Tripal 3 to Drupal 8)

      2. Chado Updates

      3. Tripal file module development

      4. Genome Project/Assembly pages

    • Request for an afternoon for a TAC meeting (perhaps Friday afternoon)

  4. File Module Request for Input  (Sean Buehler)

    • Original document

    • Recent document

    • Original Scope

      1. To automatically create download links on data pages.

      2. To provide access to files via web services.

        1. For file access (where are the files and downloading them)

        2. Random access to the file.

      3. To allow for data retrieval from indexed files for data that is not conducive for storage in Chado (e.g. genomic alignments in BAM files).

      4. Provide access control for files.

    • New ideas

      1. To be implemented as a "File" field to be associated with any given bundle type

      2. Manage both local and remote files (HTTP or FTP)

      3. Enable (or provide) file housekeeping

        1. Periodically check that all listed files are available/correct (md5?)

        2. Some kind of version control

      4. Interface with the loaders - when a file is loaded, also gets stored as a flat file

        1. Lacey:  add files uploaded via loaders to be re-used in an "existing files" drop down.

      5. Enforce standards

        1. file organization structure (IE folders) and/or standard file names

        2. file format checking (lightweight)

        3. DOI and licensing for files

  5. Training MegaSearch Module (Chun-huai Cheng)

    • Repository: https://gitlab.com/mainlabwsu/tripal_megasearch

    • Supports Drupal 7 (tripal 2 or 3)

    • Allows for download of data from chado and searching

    • You can limit the number of results a user can download

    • There is a single page where the user can choose the data type they are interested in. Then they are given a form specific to that data type that they can use to filter the results

    • Counts of results are shown before download and can be refreshed based on filters

    • You can change the output by selecting different fields

    • Some data types (e.g. genes) offer download of fasta file format. All support tab-delimited (excel).

    • Publication has a dynamic query form which can be built up one field at a time to restrict to results

    • Installation instructions are on the Readme

    • You can either serve data from the chado base tables or from the materialized views distributed with this module.

    • Static forms work best with the materialized views (some predefined with module)

    • If there is no static form available then it will fall back to the dynamic form.

    • There is a configuration form at Mainlab > Tripal MegaSearch

      1. Includes download limits, instruction text, remove duplicates, number of rows, disable view result, etc.

    • You can set up support for custom data types in the module/conf directory

      1. Example with documentation: https://gitlab.com/mainlabwsu/tripal_megasearch/blob/master/conf/tripal_megasearch.mviews.inc#L3-27

      2. Key of definition array matches to the underlying database table. You can then define which fields are available for the user to filter on in the ‘fields’ key. Specifically the key is the column name and the value is the label on the site.

    • Only supports public data. Make sure that the materialized view does not populate with private data
Tripal User's Meeting 2020/02/04 Tripal Community Meeting Notes

Community Highlights

Please add anything you would like to highlight about your Tripal-based projects: publications, new extension modules, new collaborations, newly funded grants, project update, etc.  We won't talk about these in the meetings but may tweet them from the Tripal twitter account if you approve!

Name

Highlight Description

Can we tweet it? Yes/No
     

 

Agenda

  1. Messages from the PMC (Lacey Sanderson, Bradford Condon, Stephen Ficklin)

    • Thanks to those at the Codefest who were involved in the Tripal v4 working group and who have volunteered effort to submit code updates.

  2. Messages from the TAC Chair (Meg Staton)

    • At PAG meeting had discussion about how to expand 

    • White paper and movement on grants.

  3. Report from PAG Workshop & Hackathon Committee (Lacey Sanderson, Jill Wegrzyn, Chris Childers)

    • Workshop:

      • There was a great panel of talks! We would like to extend a huge thank you to the speakers and those of you able to attend.

    • Hackathon:

      • Thank you to everyone who was able to participate either in person or remotely!

      • Very productive producing a number of merged PRs to Tripal4 and Tripal Galaxy extension module & a great community discussion regarding storage of genome assemblies resulting in a concrete plan.

  4. Training: VCF Filter Module (Lacey Sanderson)

  5. To get involved with Tripal v4 development (Lacey)

  6. Question by Abdullah: what is the migration process from v3 to v4?

    • Answer (Lacey): it's a bit too early, but we are working to make the transition as friendly as possible and our hope is to have a migration/upgrade path.