Yesterday we hosted an AMA about Product Management and what PM’s do at Pivotal Labs.
During the chat we covered:
– The difference between Project and Product Management
– What tools we use at Pivotal Labs
– Challenges of being a consultant
– How to manage mature products (open source specifically)
– Marketing challenges with MVPs
– and so much more!
Here’s what we chatted about in case you missed it (or came and thought nothing was going on because of a Hipchat feature we misunderstood). The questions have been organized to optimize readability. If you’re interested in the full transcript, email me firstname.lastname@example.org
Q: I’ve worked in the consultancy world for a long time before heading into gov, working with start up to help define mvps and drive product. I would love to know a few things. I’ll start with a cheat, What is the one question you are sick of being asked?
A: What does a Project Manager do? ;)
Q: Haha. I take it you get that a lot?
A: Yup. Also, I get a lot of … yeah… ummm so what does a Product Manager do exactly… we have one, but I don’t really get it…
I tend to say that a Product Manager is the person who figures out what to do next and how to tell everyone what’s going to happen and why.
Q: How do you answer those kinds of questions? (presuming leaving the room isn’t an option)
A: When someone asks me about project management, I help them understand the difference between PM + PM. Project management is the management of resources and deadlines. It’s following up on tasks to make sure they get done.
Part of being a Product Manager is doing the stuff a Project Manager does, but only as much as it fits in with making sure the Product gets completed per the design + timelines they are communicating. On a good self-organizing team, a project manager isn’t needed, but a product manager still sets direction and priority.
Q: What is the one thing you think even seasoned PMs miss in their day to day which could make their lives functionally better?
A: I think PMs forget to walk away from their computers and actually talk to people. Whether that’s the devs or designers, or other PMs. It helps both to create empathy for each other, and sometimes results in getting new ideas
Q: Is there a lot of consultation involved in being a PM? Like making suggestions based on your expertise about which of the clients’ ideas should be pursued or tweaked?
A: Being a PM at Pivotal involves consulting with clients to help them through the process of getting to a quality answer. It’s a lot of finesse.
Q: Do you have paired pm’s as well?
A: When there is a client Product Manager, a Pivotal PM pairs with them. We pair on writing user stories (clients learn how to make them small, acceptable, etc. )
Then we pair on prioritizing the backlog. Also, as the project progresses we pair on communicating to stakeholders etc. about what’s being built and why.
A lot of what we do as a consulting PM has to do with asking questions that help our clients evaluate their options and make decisions.
Q: How much does pairing with clients help or hurt the process of product development?
A: In a traditional PM role, you get to set direction, at Pivotal Labs, our clients do. So as a Pivotal PM, we walk clients through exercises to guide prioritization. Sometimes you end up at the same place, sometimes you end up someplace else. In the end, our clients learn how to listen to their users, evaluate risk, and respect developer insights… and that’s pretty awesome.
Q: Is 100% embedded the way to go?
A: Pairing for developers is a no brainer to me. At some point, the code will be owned by our clients, they need to know it in and out. For design + product, we don’t pair 100% of the time. We get together to discuss the stories, the priority and more.
Pairing is a way of learning by doing. We try to keep to : “I do, you watch > You do, I help > You do, I watch” for each part of being a PM.
Q: What makes a good client for Pivotal? How do you filter good from bad?
A: The best clients we’ve worked with are open to change. They are looking to build faster and smarter. They are looking to have their teams collaborate more and to get to better decisions faster. We don’t really classify anyone as a “bad” client, but not being open to working as a team is generally really hard to overcome.
Q: I love Pivotal Tracker! Do you get to work on it?
A: Pivotal Tracker is a tool that Pivots built over time to meet the needs of agile software development. It is now managed by a dedicated team in the Pivotal Labs Denver office.
Q: What else does Pivotal Labs do besides Pivotal Tracker?
A: Pivotal Labs is a design and development consulting firm. We partner with our clients (from super small startups to large enterprises) to build great products. Along the way we show our clients how to use agile and lean practices to make decisions and develop working software.
Q: What’s the ‘software stack’ of a PM at Pivotal? What tools do you use to get work done?
A: That depends on the project. Each client has different needs. We evaluate all of the tools available for each project and try to make the best decisions based on what we know at that point.
Q: Do you have internal favorites though? Like Trello for prioritizing work etc.
A: Ah! I thought you meant the developer kind of tools. We predominantly use Pivotal Tracker for project management. Occasionally we use other tools Lean Validation boards, Kanban boards, Trello, Asana, etc. for managing parts of a project, but only when the team feels it’s warranted and not for the software development side, more for design/product prioritization. Tracker is the source of truth for what will be developed/has been developed.
Also something to note when it comes to tools, is that different parts of projects require different needs. For instance, you don’t spend the time investing a user engagement dashboard to visualize how people are using the product until it launches :)
Q: Since Pivotal Tracker one of your own products, is it solely managed by the PM or is there transparency for the whole team to jump in & see what’s going on within it?
A: if we were only collaborating on design, we’d probably use something lighter, but Tracker is really great for managing agile software development.
Their backlog is private to the public, but a lot of Pivots can see it (transparency).
We all put in feature requests via the feedback button. Though our votes don’t get any special treatment :) Like any other product, the PM prioritizes feature requests.
Q: True democracy! :)
Q: You must have to deal with a lot of different personalities and approaches to working on a project. How do you deal with conflict or differences of opinion?
A: I personally try to get people to talk about why they are feeling frustrated. Then I am constantly getting people back to talking about the goals. When people can align on the goals, there is less conflict about how to get there as there is mutual respect.
Q: Hey there, thanks for conducting this session. Have a question. Hope, you can chime in. Pivotal does lot of interesting things in the open source. So, wonder if you can care to share a little about PM’s role within such open source communities – how do you help prioritize in such environments ?
A: The “Product Managers” for most of our open source tools are actually engineers. Their role there is to look through the Pull requests and comments, as well as competitive open source tools to see what the dedicated team wants to release as features and not have the community work on. Then it’s also up to them to figure out what to merge.
Q: In open source env, my take is you get contributions from different folks – to scratch their needs. so, it is interesting how do you take contributions from different ends and still avoid making the product a kitchen sink?
A: I haven’t personally managed one, but I would advise to use the same practices as any other prioritization for a product. 1) Learn about user behavior (in this case developers) and see where there are pain points 2) Figure out if how to alleviate those pain points and do the ones that make the most sense based on the goals of the product.
I’d still focus on the goals of the product. If people are contributing things that seems more like kitchen sink and will bloat the product, you may have to reject the merge (in a respectful way of course).
We have a mantra we use for mobile app development that applies to most things: “An app for everyone is an app for no one”.
That being said, if there is a big push in a open source community to add pieces you feel are not part of the core, that may also be a problem in #1, which was learn from the users. Maybe the tool is being used in high demand in a way you didn’t originally anticipated. Maybe the power users represent a different opportunity. At that point it’s time to evaluate whether to add the stuff to the first product or separate it into a second one.
Q: I couldn’t agree more on this key line – “An app for everyone is an app for no one” – +1. great response.
A: Thanks. The most recent client I worked with made a big poster with it and hung it in their office.
Q: Just with any industry – end users can be split into different categories – I would imagine very few of them are power users. Most of the times, users use 20% of your feature list. isn’t it?
A: Recently, Nic Werner (a Pivotal PM in SF) and I were talking about Power users. I tend to say work on keeping the 80% happy and occasionally throw some features the way of the Power users to delight them.
But during my conversation with him, we talked about the importance of looking at Power user behavior and seeing if the functionality they use would also be helpful to more users, but is too hidden and needs to be exposed. Then A/B testing where you have access points to that can help guide further decisions.
Q: Why not focus on the latent needs by observing the 80% average users behavior?
A: That’s my normal plan, learn from the 80%… but sometimes that features that power users are employing everyone would actually really love to use, they just don’t realize they are available
For instance, I’m sure that the reports within Pivotal Tracker are used by a small percentage of users. But, the information in them is very valuable.
I know that Dan over in Denver is trying to figure out how valuable they are to users and whether or not giving them more billing is a good idea. Only by studying user behavior in different situations can we know for sure.
Q: Most of the time, a feature gets added to scratch an itch – but PM need to focus on the hidden costs and go after the motivation. this way, there is clear justification on why this feature exists.
A: Yes, I often quote Joanna (another PM at Pivotal) who is really excellent at hearing new ideas from stakeholders and saying “Wow, that’s a great suggestion. I’ll see if there’s data to support that the users want that kind of feature.”
Q: Sometimes, I am not successful in making this happen though.
A: Reminding people that data and learning should drive decisions helps them resist scratching the itch as you called it. Or at least encourages them to validate their hunch before investing development hours in it.
Q: A PM can get to this for a proprietary product.. where you play a role in the backlog. In open source products (especially mature products such as PHP), lots of developers contribute patches – that makes sense for their context only. A PM’s job is very tricky in such environment. Keeping the overall product vision and still support active participation from community is hard to balance.
A: That is a hard thing to manage, for sure. One option maybe to split off some of the lesser used portions into smaller buildpack-esque products that can have their own product goals.
Q: I would like to learn from you about bringing developers to understand customer’s pain points discussion? Do you have developers participate in most of them or do you moderate such meetings?
A: If there is regular user testing going on, we invite the developers to take part if they want to. We always make sure to share the observations we have with the developer team. This allows for them to suggest potential fixes that often are the one that the designer/product team chooses in the end.
On a more basic level, as a PM if there is a poor user interaction that I’m noticing, I’ll demo it to the developers. A picture is worth a thousand words, I don’t know how many watching a user flow is worth.
It’s important to remember that developers are people too … and that they are also potential users. They can often be empathetic to users’ needs and pains if they are shown what’s going on.
We try to make sure the entire team understands the personas we are building the product for so that when we go over user stories, the value being delivered is well understood.
Q: What are the major requirements to becoming a PM?
A: @JoannaBeltowska wrote a great piece on this recently: bit.ly/1CnfNZZ
Two things come to mind: One is ability to be empathetic with users. Not just listening, but really understanding where there is pain. The second is the ability to communicate (with developers, stakeholders, designers etc.)
Q: Just curious – do you travel a lot or you work with the client PM over video tools such as Hangout?
A: We encourage collocation, so most of our clients actually come to our offices to work. That doesn’t always happen though, so we use hangout, skype, facetime etc. to make it as close to having everyone there as possible.
Simon (PM in NYC) recently wrote a post about using persistent hangouts when teams are remote : https://blog.pivotal.io/google-hangout-hop/
The most important thing is to make sure that remote team members stay engaged as if they were actually there in person. A problem we often encounter with our more corporate clients is the restrictions that their IT teams put on using video chat tools. Like any tool choice, we choose whatever works for the team and project.
Q: Do you use social media a lot in your role and if so what channels?
A: We don’t use social media for project specific stuff, but we do tweet and blog about product related topics. We also have a medium collection.
Even with social media, we encourage our clients to set goals for engagement and to follow the metrics on what they are doing to understand the impact.
Occasionally, we use social media as method of communication around a product and then of course there are the products we’re helping clients with that are integrated with social media platforms. If we weren’t consumers of those tools, we wouldn’t know how to design features around them.
Q: When you set out to write user stories – does your client have a clear vision on what to achieve at the end or you typically work your way by walking through the problem space etc. ?
A: We typically work through the problem first and generate a lean hypothesis around the solution. As a consultant, we’ll never know as much as our client does about their space, but we try to learn a lot (especially up front) so that we can advise to the best of our abilities.
We ask a lot questions to help understand the problem and create a list of goals.
We describe personas and user flow associated with the problem / solution and then finally we write stories.
I try to prioritize stories around what risky assumptions need to be validated and how to build something at a low cost to learn. But sometimes that’s no development at all and just a design that we show.
I don’t like to have a big icebox of user stories waiting to be put into the backlog.
I like to have 2-3 weeks worth of worth written up and then look at user feedback/ the initial goals to guide where i should invest my time in writing up the next batch of stories.
Q: Most of the time, is showing design is the best way to validate the assumption?
A: I wouldn’t said design / functional software is better at validating assumptions. Each has to be evaluated to see what the fastest way to learn is. If you have an assumption about user over a long period of time, design can rarely get you to goo understanding, but a functional tool that is lightweight often can.
Q: How do you strike the balance between MVP incremental product development and product marketing. It’s most exciting to market something brand new but if it’s too MVP or bare-bones it can be tricky to launch. As a PM is that something you take into account when you plan a product road map?
A: Totally! When there’s an MVP product, it’s important to make sure that’s understood by the users.
Sometimes that means you do a soft launch in a particular market or only to a subset of users. That allows you to understand what about the feature/product resonates most with people and how to potentially promote it (even if it’s limited functionality) to the general public.
I try to make sure the MVP is a VIABLE functional product that is valuable to a subset of users. MVP is often thought of as bare bones and if it’s too bare, it’s not viable. It has to be enough that someone would want to use it and say… I don’t want you to take it away from me.
Q: It feels like there can be a big gap between viable and marketable.
A: I’ve occasionally seen products launch with the bare bones level of multiple features. What ends up happening is that they get ripped up in the press. I think it’s better to build an MVP product with a certain level of quality around a single feature set, launch with that and talk about what’s coming next.
If you have a full feature set, that can be marketed, if a product is too lightweight across all features, marketing it is too hard. Like any good product, an MVP should deliver user value. If you market towards that user need, sometimes you can still have a successful launch.
Sorry if it comes off as, yes I try to care, but sometimes marketing is still hard for MVPs.
Q’s comment: I guess the extent to which a PM takes marketing considerations into account when thinking about an MVP depends on the PM.
Q: What about products or features where your company is playing “catch up” as the feature is considered to be table stakes for any company in that space?
A: I like the “You asked for it, We delivered” sort of messaging.
Parity features are hard to promote. But, you’re not looking to be the competitor product, you’re looking to add the value your users need. They should be happy that you’re worried about what they want and developing it.
It’s not as much about catching up to the rest of the industry as fulfilling a need.
Check out this NYTimes article about Bank of America finally allowing depositing of checks in their app. They still got press :) And people learned about the ability to do something they wanted.
I mean it took them forever to catch up to Chase and they didn’t put out a separate ad campaign around it, but they started to include it in advertising around their full functionality.
Also to note, is that you’re also prioritizing what features you’re choosing to copy from competitors based on your user needs. You’re not copying everything, just what they want.
Q: The particular example I have in mind is joint accounts functionality for the automated investing service I work for. This is a particularly sticky issue because our customer development suggests that “joint accounts” mean something different for different customers…
A: I had a similar problem recently with a client whose customers wanted “connected apps”. That means different things to different people.
Conduct user interviews, show them a few options about how you are interpreting “Joint accounts” and see where the problem really is. (Assuming you did this which is how you know there’s a difference across users).
Q: I surveyed customers to figure out what joint accounts meant to them and it’s a 40/60 split between “managing my spouses’ account from my interface” and “having some accounts (goals) in both our names that we can each access separately via our own interface”. When it’s that close to even, what would you do?
A: If you can get to the heart of the pain point, which seems to be deciding for each account/goal who can see + manage it you can make a more universal feature. Then you can choose not to use confusing terms (if understanding is split).
Choose a word that isn’t joint, but focused on the benefit you are creating. I don’t think Shared is correct either, but get into a room with some folks and chat about it. You never know what word will come out.
Q: That’s a good idea but it’s also an industry term so we want to make sure people are aware that we offer them, especially as they compare us to competitors.
A: Those check marked competitor product comparison sheets can be hard. If the industry term is not understood by all of your customers though, I’m not sure it’s that great though :)
I think my favorite is when a company invents a new term and then none of their competitors have it anymore. It’s a common marketing ploy. Make the competitors look like their are missing check boxes, simply by highlighting the product features that are unique to them.
Q: Comparison sheets help though they aren’t adapted to all communications.
A: So where are customers looking to see something that says #8220;Joint Accounts”?
Could it be accomplished in an FAQ? (once the feature is rolled out)
If you’re planning a marketing blast (emails, landing pages, tweets, etc.)
Maybe just make sure you use the industry term with the benefit to users together?
Q: I actually don’t think they are looking for it anywhere because they assume we offer it already… but when we launch the feature we may have some marketing around it. Announcement email, banner ad, tweet, potentially press…
I’d A/B/C test some samples of those with the industry terms, benefit words, and industry terms + benefit terms to see what resonates. Then use the winner for the full campaign.
Maybe a few tweets for each that says “Tell us how you use your ……..”
See which get the most replies and then use that term in the blast.
Q: Hm… I never thought to use twitter for this. Do you think there’s a risk people will be confused?
A: Confused by what? Oh that the feature isn’t there yet
Q: Yes and that we are speaking about different but related concepts (e.g. joint accounts / householding / family view)
A: hmmm. Still seems related to sharing and permissions to me. If your feature doesn’t fit perfectly into the industry “accepted” terms, I’d probably steer away from them.
There was a old flowchart on fixing your computer.
Look for the action word around what you’re trying to do, then click on it.
If you can’t find it, look for the synonyms to the action word you’re trying to do, and click on that.
If people assume you have the functionality, maybe the blast just shows the button with the call to action and says “Now you can share your goals and accounts with the people important to you.”
Don’t give it a name, just talk about how it might get used.
(obvious need for some marketing jazziness in my line above).
Q: I like the idea
A: Glad to help.
Sometimes we write product briefs on epic level features. They include 2-3 bullet points on why and why we’re going to build. They can be good guidance for marketing around what to promote once it’s ready for prime time.
Q: All of our features and changes are pretty big because we have to do them across all of our apps/clients: Web, Mac, Windows, Linux, Android, and iPhone! How would you manage releases?
A: Ah! That would frustrate me. I like being able to deliver incremental value.
I was recently discussing this with a client of ours. We were chatting about how often to do an app store release for their mobile app. He feared that offering updates too often makes it seem like they were fixing bugs and that people would get annoyed.
Q: What did you land on? I think in general, people like it when the bugs in their software that they use everyday get fixed! But yes, frequent updates are a bit annoying.
A: I said that if you’re adding value, people will appreciate it. That if you’re developing features that don’t add value, stop. But, that given the other concern (that product was slanted towards an older population that in his mind fears change), finding a balance between the two was ideal.
Q: We aim for 2 weeks across most clients. Sometimes faster, sometimes slower just depending on what’s in that iteration and what hot issues are out at the moment. Great points.
A: 2 weeks is close to what we ended up aiming towards. He was originally thinking quarterly release and I was advising for much more frequent. I think they’ll probably land on 2-4 week public releases over time, but that will be their decision.
I’d say that’s actually one of the hardest things about being a Pivotal PM.
Q: No sense in an artificial schedule driving the roadmap or backlog, obviously, but it does force you to think in smaller chunks and redefine what “value” is.
A: We tend to come from a startup / agile / lean mentality of getting something into the hands of users to learn. Often some of our more established companies can’t do that as easily because they fear it may tarnish their brand. They only want to release once something is fully polished.
Q: That would be tricky to navigate I bet.
A: We get them to a place where something isn’t perfect but meets their standards for a subset of their users and get to release it in a beta program or soft launch.
Sometimes it takes a while to get them into the mindset of gathering real user feedback as a priority. It’s incredible how quickly they buy in once they watch a few user interviews.
We all learn so much from watching real people use our products. They get addicted to user feedback and eventually metrics and then they are more willing to release to learn.
Q: As PMs at Pivotal, how do you encourage your clients interact and iterate around customer feedback to ensure the beta version is spot on?
A: Oddly one of problems tends to be that we have to remind them that a single data point (one user) shouldn’t dictate too much of a change in direction. We encourage using user feedback to guide UX and alleviate usability issues, but to use other generative research techniques (ethnography, data studies, etc.) to guide feature level direction.
Also, we don’t think most beta versions are spot on. They are a better version of a prototype that we hope to learn from.
We’re amazed as to how many companies (from startups to large enterprises) still aren’t including user feedback into their development process. Many of them still wait to build build build, release and then wait for feedback.
Tight loops during the development can really guide product direction as assumptions are validated and hypothesis are tested. It can also really help guide the designers to understand the usability challenges.
The second a client sees a user falter in a place they didn’t anticipate or say something contrary to their initial belief, they immediately realize the benefit in capturing feedback early and often.
Not much convincing needs to happen after that :)
Q: I’ve heard that at Pivotal Labs, you do mostly consulting for other PMs on how to be a great PM. Curious about how much of the research, feedback loops, and data collection you’re able to facilitate yourself and how much you’re acting as more of a mentor?
A: How we work came up earlier too. A lot of the techniques are new to the PMs we work with. We operate in the following process:
1) I do, you watch
2) You do, I help
3) You do, I watch
or I do > We do > You do
Depending on the familiarity a client PM has with any tool or technique we start with 1,2, or 3. Then repeat until everything is at a 3.
In the beginning of an engagement with a client, most things are at a 1/2 so we’re heavily involved. Over time it’s much more our clients doing and we’re watching while encouraging smaller changes to improve execution.
Sometimes our client PMs are already great. I recently worked with one in Chicago that after 2 weeks she was doing it all and I was mainly there to help with the one thing she needed help with (technical stories because she wasn’t technical at all).
Q: Do you encourage clients to adopt some of your cultural practices as well?
A: Aside from agile and lean practices around product development. We do encourage our clients to be transparent (something we really value) and to collocate teams when possible.
Recently we’ve had a strong push towards feedback internally and we’ve been trying to get more clients on board with that as well.
Feedback is a touchy subject, especially at larger organizations where people fear things being put into writing and being part of annual review processes around raises…
Q: Haha, I meant snacks, food, ping pong, open desks.
I think when clients visit our offices and work out of them for a few months, it’s hard to not want your office to have unlimited snacks, drinks, beer, and a place to hang out.
Most of those are cost concerns for companies.
We try to show them how it can help them attract talent and also to keep employees happy, improve morale, etc.
We have tuesday tech talks and a bunch of lunch activities for people. We encourage non-work related interaction too.
This reminds me of a tech talk someone from Paperless Post gave to the NYC office. He talked about three stories of scaling and the first one was about coffee.
They calculated that by buying a quality coffee machine and good coffee for the office that the time saved for productivity of people not leaving the office to go to Starbucks was huge!
Q: How often do you all host this type of AMA?
A: This is the first time we’ve done it.
We also run a Product Office hours event at our NYC, SF, and London offices. It lets startups come in for a free hour of product consulting. It’s super fun to give feedback on a small scale.
Note: We were lucky enough to have a PM from Hipchat involved in our conversation. He was there to watch us as users interact with his product in a novel way. He observed some issues that we encountered and will have to decide whether features should be developed to assist or if our public use of Hipchat is not part of the core of the app. Stay Tuned!
We learned a bit in the process about how Hipchat public rooms work with guest access (it turns out if you join mid-conversation that you can’t see the history). We have a hypothesis that a bunch of people joined and thought nothing was going on and then left. Sorry ’bout that.
And again if you’re interested in the full transcript, email me email@example.com
About the Author